Oct 25 18:54:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn systemd[1]: Starting Kubernetes Kubelet...
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: Flag --address has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: Flag --cloud-provider has been deprecated, will be removed in 1.25 or later, in favor of removing cloud provider code from Kubelet.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: Flag --pod-infra-container-image has been deprecated, will be removed in a future release. Image garbage collector will get sandbox image information from CRI.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.286007    4293 server.go:203] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288252    4293 feature_gate.go:227] unrecognized feature gate: SigstoreImageVerification
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288267    4293 feature_gate.go:227] unrecognized feature gate: VSphereStaticIPs
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288272    4293 feature_gate.go:227] unrecognized feature gate: AlibabaPlatform
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288276    4293 feature_gate.go:227] unrecognized feature gate: AzureWorkloadIdentity
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288281    4293 feature_gate.go:227] unrecognized feature gate: BuildCSIVolumes
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288285    4293 feature_gate.go:227] unrecognized feature gate: CSIDriverSharedResource
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288289    4293 feature_gate.go:227] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288294    4293 feature_gate.go:227] unrecognized feature gate: PrivateHostedZoneAWS
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288298    4293 feature_gate.go:242] Setting GA feature gate RetroactiveDefaultStorageClass=false. It will be removed in a future release.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288302    4293 feature_gate.go:227] unrecognized feature gate: AdminNetworkPolicy
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288306    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProvider
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288311    4293 feature_gate.go:227] unrecognized feature gate: GCPLabelsTags
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288314    4293 feature_gate.go:227] unrecognized feature gate: AutomatedEtcdBackup
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288318    4293 feature_gate.go:227] unrecognized feature gate: DNSNameResolver
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288322    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderGCP
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288326    4293 feature_gate.go:227] unrecognized feature gate: MachineAPIProviderOpenStack
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288329    4293 feature_gate.go:227] unrecognized feature gate: RouteExternalCertificate
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288333    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderAzure
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288337    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderExternal
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288340    4293 feature_gate.go:227] unrecognized feature gate: InsightsConfigAPI
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288344    4293 feature_gate.go:227] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.288348    4293 feature_gate.go:227] unrecognized feature gate: GatewayAPI
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288409    4293 flags.go:64] FLAG: --address="10.0.3.42"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288424    4293 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288430    4293 flags.go:64] FLAG: --anonymous-auth="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288434    4293 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288438    4293 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288442    4293 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288446    4293 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288451    4293 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288455    4293 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288458    4293 flags.go:64] FLAG: --azure-container-registry-config=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288461    4293 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288465    4293 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288477    4293 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288480    4293 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288483    4293 flags.go:64] FLAG: --cgroup-root=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288487    4293 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288490    4293 flags.go:64] FLAG: --client-ca-file=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288493    4293 flags.go:64] FLAG: --cloud-config=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288496    4293 flags.go:64] FLAG: --cloud-provider="external"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288499    4293 flags.go:64] FLAG: --cluster-dns="[]"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288504    4293 flags.go:64] FLAG: --cluster-domain=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288507    4293 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288510    4293 flags.go:64] FLAG: --config-dir=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288513    4293 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288517    4293 flags.go:64] FLAG: --container-log-max-files="5"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288521    4293 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288524    4293 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288527    4293 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288531    4293 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288534    4293 flags.go:64] FLAG: --contention-profiling="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288538    4293 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288541    4293 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288544    4293 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288548    4293 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288554    4293 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288557    4293 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288560    4293 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288563    4293 flags.go:64] FLAG: --enable-load-reader="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288566    4293 flags.go:64] FLAG: --enable-server="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288569    4293 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288573    4293 flags.go:64] FLAG: --event-burst="100"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288576    4293 flags.go:64] FLAG: --event-qps="50"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288579    4293 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288582    4293 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288585    4293 flags.go:64] FLAG: --eviction-hard=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288589    4293 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288592    4293 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288595    4293 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288598    4293 flags.go:64] FLAG: --eviction-soft=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288601    4293 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288606    4293 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288609    4293 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288612    4293 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288615    4293 flags.go:64] FLAG: --fail-swap-on="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288618    4293 flags.go:64] FLAG: --feature-gates=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288622    4293 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288625    4293 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288628    4293 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288631    4293 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288634    4293 flags.go:64] FLAG: --healthz-port="10248"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288637    4293 flags.go:64] FLAG: --help="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288640    4293 flags.go:64] FLAG: --hostname-override="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288644    4293 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288647    4293 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288650    4293 flags.go:64] FLAG: --image-credential-provider-bin-dir=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288654    4293 flags.go:64] FLAG: --image-credential-provider-config=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288657    4293 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288661    4293 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288664    4293 flags.go:64] FLAG: --image-service-endpoint=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288667    4293 flags.go:64] FLAG: --iptables-drop-bit="15"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288670    4293 flags.go:64] FLAG: --iptables-masquerade-bit="14"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288673    4293 flags.go:64] FLAG: --keep-terminated-pod-volumes="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288676    4293 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288679    4293 flags.go:64] FLAG: --kube-api-burst="100"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288682    4293 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288686    4293 flags.go:64] FLAG: --kube-api-qps="50"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288689    4293 flags.go:64] FLAG: --kube-reserved=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288692    4293 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288695    4293 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288699    4293 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288702    4293 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288705    4293 flags.go:64] FLAG: --lock-file=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288708    4293 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288711    4293 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288715    4293 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288720    4293 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.288723    4293 flags.go:64] FLAG: --logging-format="text"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289712    4293 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289720    4293 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289723    4293 flags.go:64] FLAG: --manifest-url=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289726    4293 flags.go:64] FLAG: --manifest-url-header=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289731    4293 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289734    4293 flags.go:64] FLAG: --max-open-files="1000000"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289738    4293 flags.go:64] FLAG: --max-pods="110"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289741    4293 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289744    4293 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289747    4293 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289750    4293 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289753    4293 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289757    4293 flags.go:64] FLAG: --node-ip="10.0.3.42"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289760    4293 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289768    4293 flags.go:64] FLAG: --node-status-max-images="50"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289772    4293 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289775    4293 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289779    4293 flags.go:64] FLAG: --pod-cidr=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289782    4293 flags.go:64] FLAG: --pod-infra-container-image="registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:aa40faef7732804a99271483a83076a9573c8ecc6e029f7cdd2b4faa7bede925"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289788    4293 flags.go:64] FLAG: --pod-manifest-path=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289792    4293 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289795    4293 flags.go:64] FLAG: --pods-per-core="0"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289799    4293 flags.go:64] FLAG: --port="10250"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289802    4293 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289805    4293 flags.go:64] FLAG: --provider-id=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289807    4293 flags.go:64] FLAG: --qos-reserved=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289810    4293 flags.go:64] FLAG: --read-only-port="10255"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289813    4293 flags.go:64] FLAG: --register-node="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289816    4293 flags.go:64] FLAG: --register-schedulable="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289819    4293 flags.go:64] FLAG: --register-with-taints=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289823    4293 flags.go:64] FLAG: --registry-burst="10"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289826    4293 flags.go:64] FLAG: --registry-qps="5"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289829    4293 flags.go:64] FLAG: --reserved-cpus=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289832    4293 flags.go:64] FLAG: --reserved-memory=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289836    4293 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289839    4293 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289842    4293 flags.go:64] FLAG: --rotate-certificates="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289845    4293 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289848    4293 flags.go:64] FLAG: --runonce="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289851    4293 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289854    4293 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289857    4293 flags.go:64] FLAG: --seccomp-default="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289860    4293 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289863    4293 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289866    4293 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289869    4293 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289872    4293 flags.go:64] FLAG: --storage-driver-password="root"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289875    4293 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289878    4293 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289881    4293 flags.go:64] FLAG: --storage-driver-user="root"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289884    4293 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289889    4293 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289892    4293 flags.go:64] FLAG: --system-cgroups=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289895    4293 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=500Mi"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289900    4293 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289904    4293 flags.go:64] FLAG: --tls-cert-file=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289907    4293 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289912    4293 flags.go:64] FLAG: --tls-min-version=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289915    4293 flags.go:64] FLAG: --tls-private-key-file=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289919    4293 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289922    4293 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289925    4293 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289928    4293 flags.go:64] FLAG: --v="2"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289933    4293 flags.go:64] FLAG: --version="false"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289937    4293 flags.go:64] FLAG: --vmodule=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289941    4293 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.289945    4293 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.289999    4293 feature_gate.go:227] unrecognized feature gate: AlibabaPlatform
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290003    4293 feature_gate.go:227] unrecognized feature gate: AzureWorkloadIdentity
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290007    4293 feature_gate.go:227] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290011    4293 feature_gate.go:227] unrecognized feature gate: PrivateHostedZoneAWS
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290014    4293 feature_gate.go:227] unrecognized feature gate: BuildCSIVolumes
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290018    4293 feature_gate.go:227] unrecognized feature gate: CSIDriverSharedResource
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290022    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProvider
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290026    4293 feature_gate.go:227] unrecognized feature gate: GCPLabelsTags
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290038    4293 feature_gate.go:242] Setting GA feature gate RetroactiveDefaultStorageClass=false. It will be removed in a future release.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290042    4293 feature_gate.go:227] unrecognized feature gate: AdminNetworkPolicy
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290046    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderGCP
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290049    4293 feature_gate.go:227] unrecognized feature gate: AutomatedEtcdBackup
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290053    4293 feature_gate.go:227] unrecognized feature gate: DNSNameResolver
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290057    4293 feature_gate.go:227] unrecognized feature gate: MachineAPIProviderOpenStack
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290060    4293 feature_gate.go:227] unrecognized feature gate: RouteExternalCertificate
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290064    4293 feature_gate.go:227] unrecognized feature gate: InsightsConfigAPI
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290068    4293 feature_gate.go:227] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290072    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderAzure
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290075    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderExternal
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290079    4293 feature_gate.go:227] unrecognized feature gate: GatewayAPI
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290083    4293 feature_gate.go:227] unrecognized feature gate: VSphereStaticIPs
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.290088    4293 feature_gate.go:227] unrecognized feature gate: SigstoreImageVerification
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.290092    4293 feature_gate.go:250] feature gates: &{map[CloudDualStackNodeIPs:true DynamicResourceAllocation:false EventedPLEG:false MaxUnavailableStatefulSet:false NodeSwap:false RetroactiveDefaultStorageClass:false ValidatingAdmissionPolicy:false]}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.295691    4293 server.go:467] "Kubelet version" kubeletVersion="v1.28.2+d981348"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.295721    4293 server.go:469] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295761    4293 feature_gate.go:227] unrecognized feature gate: AdminNetworkPolicy
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295768    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProvider
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295772    4293 feature_gate.go:227] unrecognized feature gate: GCPLabelsTags
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295777    4293 feature_gate.go:242] Setting GA feature gate RetroactiveDefaultStorageClass=false. It will be removed in a future release.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295783    4293 feature_gate.go:227] unrecognized feature gate: AutomatedEtcdBackup
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295788    4293 feature_gate.go:227] unrecognized feature gate: DNSNameResolver
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295793    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderGCP
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295797    4293 feature_gate.go:227] unrecognized feature gate: MachineAPIProviderOpenStack
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295802    4293 feature_gate.go:227] unrecognized feature gate: RouteExternalCertificate
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295806    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderAzure
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295811    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderExternal
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295815    4293 feature_gate.go:227] unrecognized feature gate: InsightsConfigAPI
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295819    4293 feature_gate.go:227] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295823    4293 feature_gate.go:227] unrecognized feature gate: GatewayAPI
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295827    4293 feature_gate.go:227] unrecognized feature gate: SigstoreImageVerification
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295831    4293 feature_gate.go:227] unrecognized feature gate: VSphereStaticIPs
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295835    4293 feature_gate.go:227] unrecognized feature gate: AlibabaPlatform
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295839    4293 feature_gate.go:227] unrecognized feature gate: AzureWorkloadIdentity
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295843    4293 feature_gate.go:227] unrecognized feature gate: BuildCSIVolumes
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295846    4293 feature_gate.go:227] unrecognized feature gate: CSIDriverSharedResource
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295850    4293 feature_gate.go:227] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295854    4293 feature_gate.go:227] unrecognized feature gate: PrivateHostedZoneAWS
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.295860    4293 feature_gate.go:250] feature gates: &{map[CloudDualStackNodeIPs:true DynamicResourceAllocation:false EventedPLEG:false MaxUnavailableStatefulSet:false NodeSwap:false RetroactiveDefaultStorageClass:false ValidatingAdmissionPolicy:false]}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295933    4293 feature_gate.go:227] unrecognized feature gate: AlibabaPlatform
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295939    4293 feature_gate.go:227] unrecognized feature gate: AzureWorkloadIdentity
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295943    4293 feature_gate.go:227] unrecognized feature gate: BuildCSIVolumes
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295947    4293 feature_gate.go:227] unrecognized feature gate: CSIDriverSharedResource
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295951    4293 feature_gate.go:227] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295955    4293 feature_gate.go:227] unrecognized feature gate: PrivateHostedZoneAWS
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295959    4293 feature_gate.go:242] Setting GA feature gate RetroactiveDefaultStorageClass=false. It will be removed in a future release.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295964    4293 feature_gate.go:227] unrecognized feature gate: AdminNetworkPolicy
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295968    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProvider
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295971    4293 feature_gate.go:227] unrecognized feature gate: GCPLabelsTags
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295976    4293 feature_gate.go:227] unrecognized feature gate: AutomatedEtcdBackup
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295979    4293 feature_gate.go:227] unrecognized feature gate: DNSNameResolver
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295983    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderGCP
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295987    4293 feature_gate.go:227] unrecognized feature gate: MachineAPIProviderOpenStack
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295990    4293 feature_gate.go:227] unrecognized feature gate: RouteExternalCertificate
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295994    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderAzure
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.295998    4293 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderExternal
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.296002    4293 feature_gate.go:227] unrecognized feature gate: InsightsConfigAPI
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.296006    4293 feature_gate.go:227] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.296010    4293 feature_gate.go:227] unrecognized feature gate: GatewayAPI
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.296014    4293 feature_gate.go:227] unrecognized feature gate: SigstoreImageVerification
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.296018    4293 feature_gate.go:227] unrecognized feature gate: VSphereStaticIPs
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.296023    4293 feature_gate.go:250] feature gates: &{map[CloudDualStackNodeIPs:true DynamicResourceAllocation:false EventedPLEG:false MaxUnavailableStatefulSet:false NodeSwap:false RetroactiveDefaultStorageClass:false ValidatingAdmissionPolicy:false]}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.296185    4293 server.go:901] "Client rotation is on, will bootstrap in background"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.339105    4293 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.339366    4293 server.go:958] "Starting client certificate rotation"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.340145    4293 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Certificate rotation is enabled
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.340308    4293 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Rotating certificates
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.352395    4293 dynamic_cafile_content.go:119] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.352463    4293 util_unix.go:103] "Using this endpoint is deprecated, please consider using full URL format" endpoint="/var/run/crio/crio.sock" URL="unix:///var/run/crio/crio.sock"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.352511    4293 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.363282    4293 remote_runtime.go:143] "Validated CRI v1 runtime API"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.363311    4293 util_unix.go:103] "Using this endpoint is deprecated, please consider using full URL format" endpoint="/var/run/crio/crio.sock" URL="unix:///var/run/crio/crio.sock"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.366852    4293 remote_image.go:111] "Validated CRI v1 image API"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.371505    4293 manager.go:162] cAdvisor running in container: "/system.slice/kubelet.service"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.377877    4293 fs.go:133] Filesystem UUIDs: map[0568037f-73fb-422a-81f3-f4f96c744efb:/dev/vda4 3105-E089:/dev/vda2 ef506d8f-dd01-416d-a362-05e7d4dce2a3:/dev/vda3]
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.377902    4293 fs.go:134] Filesystem partitions: map[/dev/shm:{mountpoint:/dev/shm major:0 minor:23 fsType:tmpfs blockSize:0} /dev/vda3:{mountpoint:/boot major:252 minor:3 fsType:ext4 blockSize:0} /dev/vda4:{mountpoint:/var major:252 minor:4 fsType:xfs blockSize:0} /run:{mountpoint:/run major:0 minor:25 fsType:tmpfs blockSize:0} /sys/fs/cgroup:{mountpoint:/sys/fs/cgroup major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:44 fsType:tmpfs blockSize:0}]
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.383432    4293 manager.go:210] Machine: {Timestamp:2023-10-25 18:54:12.383010312 +0000 UTC m=+0.551740327 CPUVendorID:AuthenticAMD NumCores:8 NumPhysicalCores:1 NumSockets:8 CpuFrequency:2800000 MemoryCapacity:16759255040 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:4} {PageSize:2048 NumPages:0}] MachineID:d60d167946e84c789f1c90160e251f32 SystemUUID:d60d1679-46e8-4c78-9f1c-90160e251f32 BootID:3f433632-7802-4078-87ce-4d342f8e3480 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:23 Capacity:8379625472 Type:vfs Inodes:2045807 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:25 Capacity:3351851008 Type:vfs Inodes:819200 HasInodes:true} {Device:/sys/fs/cgroup DeviceMajor:0 DeviceMinor:26 Capacity:4194304 Type:vfs Inodes:1024 HasInodes:true} {Device:/dev/vda4 DeviceMajor:252 DeviceMinor:4 Capacity:42344624128 Type:vfs Inodes:20708800 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:44 Capacity:8379629568 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/vda3 DeviceMajor:252 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[252:0:{Name:vda Major:252 Minor:0 Size:42949672960 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:fa:16:3e:e6:c8:e6 Speed:0 Mtu:1342} {Name:enp3s0 MacAddress:fa:16:3e:e6:c8:e6 Speed:-1 Mtu:1342} {Name:enp4s0 MacAddress:fa:16:3e:73:67:7f Speed:-1 Mtu:1400} {Name:enp6s0 MacAddress:fa:16:3e:c2:2d:2a Speed:-1 Mtu:1500} {Name:ovs-system MacAddress:02:e2:cf:03:b9:4f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:16759255040 HugePages:[{PageSize:1048576 NumPages:4} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:0 Size:16777216 Type:Unified Level:3}] SocketID:0} {Id:0 Threads:[1] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:1 Size:16777216 Type:Unified Level:3}] SocketID:1} {Id:0 Threads:[2] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:2 Size:16777216 Type:Unified Level:3}] SocketID:2} {Id:0 Threads:[3] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:3 Size:16777216 Type:Unified Level:3}] SocketID:3} {Id:0 Threads:[4] Caches:[{Id:4 Size:32768 Type:Data Level:1} {Id:4 Size:32768 Type:Instruction Level:1} {Id:4 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:4 Size:16777216 Type:Unified Level:3}] SocketID:4} {Id:0 Threads:[5] Caches:[{Id:5 Size:32768 Type:Data Level:1} {Id:5 Size:32768 Type:Instruction Level:1} {Id:5 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:5 Size:16777216 Type:Unified Level:3}] SocketID:5} {Id:0 Threads:[6] Caches:[{Id:6 Size:32768 Type:Data Level:1} {Id:6 Size:32768 Type:Instruction Level:1} {Id:6 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:6 Size:16777216 Type:Unified Level:3}] SocketID:6} {Id:0 Threads:[7] Caches:[{Id:7 Size:32768 Type:Data Level:1} {Id:7 Size:32768 Type:Instruction Level:1} {Id:7 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:7 Size:16777216 Type:Unified Level:3}] SocketID:7}] Caches:[] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.383558    4293 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.383590    4293 manager.go:226] Version: {KernelVersion:5.14.0-284.36.1.el9_2.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 415.92.202310241500-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.383663    4293 server.go:745] "Option --reserved-cpus is specified, it will overwrite the cpu setting in KubeReserved and SystemReserved" kubeReservedCPUs={"memory":"500Mi"} systemReservedCPUs={"cpu":"500m","ephemeral-storage":"1Gi","memory":"500Mi"}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.383677    4293 server.go:753] "After cpu setting is overwritten" kubeReservedCPUs={"memory":"500Mi"} systemReservedCPUs={"cpu":"2","ephemeral-storage":"1Gi","memory":"500Mi"}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.384883    4293 container_manager_linux.go:268] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.385039    4293 container_manager_linux.go:273] "Creating Container Manager object based on Node Config" nodeConfig={"RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":{"memory":"500Mi"},"SystemReserved":{"cpu":"2","ephemeral-storage":"1Gi","memory":"500Mi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"static","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":5000000000,"ExperimentalMemoryManagerPolicy":"None","ExperimentalMemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"best-effort","TopologyManagerPolicyOptions":null}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.385067    4293 topology_manager.go:147] "Creating topology manager with policy per scope" topologyPolicyName="best-effort" topologyScopeName="container" topologyPolicyOptions={"PreferClosestNUMA":false}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.385077    4293 container_manager_linux.go:304] "Creating device plugin manager"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.386025    4293 manager.go:135] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.386048    4293 server.go:66] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.386159    4293 cpu_manager.go:173] "Detected CPU topology" topology={"NumCPUs":8,"NumCores":8,"NumSockets":8,"NumNUMANodes":1,"CPUDetails":{"0":{"NUMANodeID":0,"SocketID":0,"CoreID":0},"1":{"NUMANodeID":0,"SocketID":1,"CoreID":1},"2":{"NUMANodeID":0,"SocketID":2,"CoreID":2},"3":{"NUMANodeID":0,"SocketID":3,"CoreID":3},"4":{"NUMANodeID":0,"SocketID":4,"CoreID":4},"5":{"NUMANodeID":0,"SocketID":5,"CoreID":5},"6":{"NUMANodeID":0,"SocketID":6,"CoreID":6},"7":{"NUMANodeID":0,"SocketID":7,"CoreID":7}}}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.386210    4293 policy_static.go:139] "Static policy created with configuration" options={"FullPhysicalCPUsOnly":false,"DistributeCPUsAcrossNUMA":false,"AlignBySocket":false}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.386230    4293 policy_static.go:175] "Reserved CPUs not available for exclusive assignment" reservedSize=2 reserved="0-1" reservedPhysicalCPUs="0-1"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.386236    4293 state_mem.go:36] "Initialized new in-memory state store"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.386276    4293 server.go:1203] "Using root directory" path="/var/lib/kubelet"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.391177    4293 kubelet.go:402] "Attempting to sync node with API server"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.391201    4293 kubelet.go:307] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.391479    4293 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.391494    4293 kubelet.go:318] "Adding apiserver pod source"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.391504    4293 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.394427    4293 kuberuntime_manager.go:257] "Container runtime initialized" containerRuntime="cri-o" version="1.28.1-7.rhaos4.15.gitd3d1f84.el9" apiVersion="v1"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.396535    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/rbd"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.396564    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/azure-file"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.396587    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/vsphere-volume"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.396608    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.396623    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.397142    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.397163    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.397179    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.397199    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.397212    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.397221    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/cephfs"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.397234    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.397249    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.397287    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.397299    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.397310    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.397638    4293 certificate_manager.go:562] kubernetes.io/kube-apiserver-client-kubelet: Failed while requesting a signed certificate from the control plane: cannot create certificate signing request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/certificates.k8s.io/v1/certificatesigningrequests": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.397978    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.397970    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.398106    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.398110    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.400364    4293 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.404191    4293 server.go:1238] "Started kubelet"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.404235    4293 kubelet.go:1444] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn systemd[1]: Started Kubernetes Kubelet.
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.405971    4293 ratelimit.go:65] "Setting rate limiting for podresources endpoint" qps=100 burstTokens=10
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.405977    4293 server.go:162] "Starting to listen" address="10.0.3.42" port=10250
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.406224    4293 server.go:233] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.406658    4293 certificate_manager.go:356] kubernetes.io/kubelet-serving: Certificate rotation is enabled
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.406689    4293 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.407318    4293 volume_manager.go:289] "The desired_state_of_world populator starts"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.407337    4293 volume_manager.go:291] "Starting Kubelet Volume Manager"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.407582    4293 desired_state_of_world_populator.go:151] "Desired state populator starts to run"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.407643    4293 reconstruct_new.go:102] "Volume reconstruction finished"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.407656    4293 reconciler_new.go:29] "Reconciler: start to sync state"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.419306    4293 server.go:462] "Adding debug handlers to kubelet server"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.420531    4293 factory.go:153] Registering CRI-O factory
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.420553    4293 factory.go:55] Registering systemd factory
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.420638    4293 factory.go:103] Registering Raw factory
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.420700    4293 manager.go:1186] Started watching for new ooms in manager
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.421936    4293 manager.go:299] Starting recovery of all containers
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.424015    4293 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e8517c93016", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 404154390, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 404154390, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.424247    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.424385    4293 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="200ms"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.424494    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.424586    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.435238    4293 manager.go:304] Recovery completed
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.460081    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.460905    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.460940    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.460951    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.461320    4293 cpu_manager.go:215] "Starting CPU manager" policy="static"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.461339    4293 cpu_manager.go:216] "Reconciling" reconcilePeriod="5s"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.461405    4293 state_mem.go:36] "Initialized new in-memory state store"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.467226    4293 state_mem.go:88] "Updated default CPUSet" cpuSet="0-7"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.472072    4293 memory_manager.go:169] "Starting memorymanager" policy="None"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.472109    4293 state_mem.go:35] "Initializing new in-memory state store"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.506192    4293 manager.go:295] "Starting Device Plugin manager"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.506246    4293 manager.go:471] "Failed to read data from checkpoint" checkpoint="kubelet_internal_checkpoint" err="checkpoint is not found"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.506256    4293 server.go:79] "Starting device plugin registration server"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.506497    4293 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.506543    4293 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.506553    4293 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.506551    4293 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.508289    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.509021    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.509050    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.509060    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.509090    4293 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.515056    4293 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.572143    4293 kubelet_network_linux.go:50] "Initialized iptables rules." protocol="IPv4"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.574245    4293 kubelet_network_linux.go:50] "Initialized iptables rules." protocol="IPv6"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.574278    4293 status_manager.go:217] "Starting to sync pod status with apiserver"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.574294    4293 kubelet.go:2313] "Starting kubelet main sync loop"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.574385    4293 kubelet.go:2337] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:12.591786    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.591828    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.629062    4293 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="400ms"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675092    4293 kubelet.go:2399] "SyncLoop ADD" source="file" pods=["openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn","openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"]
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675145    4293 topology_manager.go:215] "Topology Admit Handler" podUID="74b8c21149c0164c4da6a5858bca8c45" podNamespace="openshift-openstack-infra" podName="coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675230    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-coredns"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675247    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-coredns"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675254    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-coredns"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675266    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675271    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675277    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675293    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675299    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-coredns"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675305    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-coredns"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675326    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675333    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675338    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675342    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675347    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675351    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675357    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675363    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675369    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675379    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns-monitor"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675387    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns-monitor"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675392    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns-monitor"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675397    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675401    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675406    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675411    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675416    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns-monitor"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675422    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns-monitor"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675442    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.675998    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676031    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676042    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676143    4293 topology_manager.go:215] "Topology Admit Handler" podUID="ef062c82f679a1ef42be9dd35e115e2f" podNamespace="openshift-openstack-infra" podName="keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676166    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-keepalived"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676191    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-keepalived"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676200    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-keepalived"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676208    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676214    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676221    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676229    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676236    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-keepalived"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676247    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-keepalived"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676263    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676272    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676280    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676286    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676292    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676297    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676302    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676307    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676312    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676322    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived-monitor"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676328    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived-monitor"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676334    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived-monitor"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676338    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676342    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676346    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676352    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676357    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived-monitor"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676362    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived-monitor"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676377    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676956    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676995    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.676960    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.677007    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.677227    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.677656    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.677686    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.677698    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.678067    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.678131    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.678145    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709575    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nm-resolv\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-nm-resolv\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709605    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"script-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-script-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709631    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfigvarlib\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfigvarlib\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709665    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef062c82f679a1ef42be9dd35e115e2f-run-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709684    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"chroot-host\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-chroot-host\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709721    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-conf-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709741    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nodeip-configuration\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-nodeip-configuration\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709758    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-resource-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709777    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-kubeconfig\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709811    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-conf-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709832    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-resource-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.709851    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfig\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.715364    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.715805    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.715883    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.715943    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.716025    4293 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:12.720732    4293 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810090    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"nm-resolv\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-nm-resolv\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810159    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"script-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-script-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810188    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kubeconfigvarlib\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfigvarlib\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810212    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"run-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef062c82f679a1ef42be9dd35e115e2f-run-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810239    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"chroot-host\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-chroot-host\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810264    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-conf-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810289    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"nodeip-configuration\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-nodeip-configuration\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810333    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-resource-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810361    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-kubeconfig\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810392    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-conf-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810426    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-resource-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810429    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"nodeip-configuration\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-nodeip-configuration\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810446    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"script-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-script-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810463    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfig\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810505    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"chroot-host\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-chroot-host\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810515    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"nm-resolv\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-nm-resolv\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810541    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-resource-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810560    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-conf-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810544    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-conf-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810566    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kubeconfigvarlib\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfigvarlib\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810501    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfig\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810625    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-resource-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810633    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-kubeconfig\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.810900    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"run-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef062c82f679a1ef42be9dd35e115e2f-run-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.994758    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:12.995351    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:13.034319    4293 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="800ms"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:13.121736    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:13.122177    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:13.122204    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:13.122215    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:13.122243    4293 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:13.126262    4293 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:13.272954    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:13.273004    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:13.312006    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:13.312086    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:13.429482    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:13.840382    4293 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="1.6s"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:13.926966    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:13.927698    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:13.927721    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:13.927730    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:13.927753    4293 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:13.932314    4293 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:13.954527    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:13.954603    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:14.041454    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:14.041482    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:14.433476    4293 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Rotating certificates
Oct 25 18:54:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:14.434088    4293 certificate_manager.go:562] kubernetes.io/kube-apiserver-client-kubelet: Failed while requesting a signed certificate from the control plane: cannot create certificate signing request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/certificates.k8s.io/v1/certificatesigningrequests": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:14.434143    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:14.450472    4293 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e8517c93016", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 404154390, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 404154390, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:15.028532    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:15.028590    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:15.430983    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:15.446601    4293 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="3.2s"
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:15.532773    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:15.533620    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:15.533665    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:15.533676    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:15.533731    4293 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:15.540533    4293 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:15.801013    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:15.801088    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:15.801143    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:15.801178    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:16.368912    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:16.368986    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:16.431292    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:17.430673    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:18.429525    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:18.652289    4293 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="6.4s"
Oct 25 18:54:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:18.741473    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:18.742143    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:18.742190    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:18.742204    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:18.742268    4293 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:18.747295    4293 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:18.775404    4293 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Rotating certificates
Oct 25 18:54:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:18.780252    4293 certificate_manager.go:562] kubernetes.io/kube-apiserver-client-kubelet: Failed while requesting a signed certificate from the control plane: cannot create certificate signing request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/certificates.k8s.io/v1/certificatesigningrequests": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:19.380068    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:19.380111    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:19.429159    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:19.574255    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:19.574298    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:19.792616    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:19.792656    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:20.430331    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:21.438871    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:21.811144    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-76de8220d96c5acc92af30294e8d2ee82176e898e60a27b0beaa0ae58ea87782 WatchSource:0}: Error finding container 76de8220d96c5acc92af30294e8d2ee82176e898e60a27b0beaa0ae58ea87782: Status 404 returned error can't find the container with id 76de8220d96c5acc92af30294e8d2ee82176e898e60a27b0beaa0ae58ea87782
Oct 25 18:54:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:21.811588    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-6bd3b6dc6cbecb3e1b3f7861279d009c432fcdb268d8bd6f76331363ba2b48ce WatchSource:0}: Error finding container 6bd3b6dc6cbecb3e1b3f7861279d009c432fcdb268d8bd6f76331363ba2b48ce: Status 404 returned error can't find the container with id 6bd3b6dc6cbecb3e1b3f7861279d009c432fcdb268d8bd6f76331363ba2b48ce
Oct 25 18:54:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:22.244574    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:22.244610    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:22.430651    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:22.507105    4293 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:22.595864    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event={"ID":"ef062c82f679a1ef42be9dd35e115e2f","Type":"ContainerStarted","Data":"6bd3b6dc6cbecb3e1b3f7861279d009c432fcdb268d8bd6f76331363ba2b48ce"}
Oct 25 18:54:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:22.596533    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event={"ID":"74b8c21149c0164c4da6a5858bca8c45","Type":"ContainerStarted","Data":"76de8220d96c5acc92af30294e8d2ee82176e898e60a27b0beaa0ae58ea87782"}
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.429498    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.600005    4293 generic.go:334] "Generic (PLEG): container finished" podID="ef062c82f679a1ef42be9dd35e115e2f" containerID="45ab12dbf1f318fbffa119c47e83cfa66fd458c79ac5d61b63bc59401d2f4dca" exitCode=0
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.600046    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.600085    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event={"ID":"ef062c82f679a1ef42be9dd35e115e2f","Type":"ContainerDied","Data":"45ab12dbf1f318fbffa119c47e83cfa66fd458c79ac5d61b63bc59401d2f4dca"}
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.600750    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.600777    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.600789    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.601527    4293 generic.go:334] "Generic (PLEG): container finished" podID="74b8c21149c0164c4da6a5858bca8c45" containerID="ed18ad9d769b455f35409e98f71c19fc16b05ee157ac677d6c3bbd445f4a269b" exitCode=0
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.601561    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event={"ID":"74b8c21149c0164c4da6a5858bca8c45","Type":"ContainerDied","Data":"ed18ad9d769b455f35409e98f71c19fc16b05ee157ac677d6c3bbd445f4a269b"}
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.601580    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.602105    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.602154    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.602166    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:23.602971    4293 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 25 18:54:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:24.428442    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:24.455905    4293 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e8517c93016", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 404154390, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 404154390, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:54:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:25.057719    4293 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:54:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:25.148152    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:25.149168    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:25.149203    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:25.149214    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:25.149238    4293 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:25.152941    4293 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:25.429367    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:26.430395    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:26.814527    4293 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Rotating certificates
Oct 25 18:54:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:26.820277    4293 certificate_manager.go:562] kubernetes.io/kube-apiserver-client-kubelet: Failed while requesting a signed certificate from the control plane: cannot create certificate signing request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/certificates.k8s.io/v1/certificatesigningrequests": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:27.106006    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:27.106073    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:27.429940    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:28.440866    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:29.430066    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:29.674039    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:29.674129    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:30.430278    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:30.613081    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event={"ID":"ef062c82f679a1ef42be9dd35e115e2f","Type":"ContainerStarted","Data":"d24877187dcf8db8f6e8e89977c3b9d7df27252361766fd67b281d2cd4d433df"}
Oct 25 18:54:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:30.613107    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event={"ID":"ef062c82f679a1ef42be9dd35e115e2f","Type":"ContainerStarted","Data":"23d105817a8804581c88eb414d73b2d89197eecc22e4069caae1b9bb9d97fa70"}
Oct 25 18:54:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:30.613188    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:30.613678    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:30.613697    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:30.613706    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:30.916951    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:30.917015    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:54:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:31.435625    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:31.619742    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:31.619799    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event={"ID":"74b8c21149c0164c4da6a5858bca8c45","Type":"ContainerStarted","Data":"ff854a423701c844bd0631ae16f9562feae507e20d0bbecaed74462886f84b44"}
Oct 25 18:54:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:31.619849    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event={"ID":"74b8c21149c0164c4da6a5858bca8c45","Type":"ContainerStarted","Data":"cd77cdebb23838e100b8e20e058052841a1f9f2c38d013c5c5d7f84c30658896"}
Oct 25 18:54:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:31.619806    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:31.620352    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:31.620389    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:31.620400    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:31.620947    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:31.620976    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:31.620988    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:32.064400    4293 controller.go:146] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.153576    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.154160    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.154185    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.154193    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.154218    4293 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:32.160275    4293 kubelet_node_status.go:94] "Unable to register node with API server" err="nodes is forbidden: User \"system:anonymous\" cannot create resource \"nodes\" in API group \"\" at the cluster scope" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:32.205826    4293 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:32.205947    4293 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.431324    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.491213    4293 state_mem.go:80] "Updated desired CPUSet" podUID="ef062c82f679a1ef42be9dd35e115e2f" containerName="keepalived" cpuSet="0-7"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.501970    4293 state_mem.go:80] "Updated desired CPUSet" podUID="ef062c82f679a1ef42be9dd35e115e2f" containerName="keepalived-monitor" cpuSet="0-7"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:32.507663    4293 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.513082    4293 state_mem.go:80] "Updated desired CPUSet" podUID="74b8c21149c0164c4da6a5858bca8c45" containerName="coredns" cpuSet="0-7"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.523415    4293 state_mem.go:80] "Updated desired CPUSet" podUID="74b8c21149c0164c4da6a5858bca8c45" containerName="coredns-monitor" cpuSet="0-7"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.620718    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.624856    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.624884    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:32.624895    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:33.432241    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:34.430733    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.462612    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e8517c93016", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 404154390, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 404154390, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.468554    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460923678, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460923678, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.474143    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460945609, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460945609, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.480518    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460955359, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460955359, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.486953    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851da6ae1f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 502556191, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 502556191, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.492526    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460923678, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 509037434, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.497973    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460945609, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 509054865, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.503711    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460955359, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 509063075, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.509525    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460923678, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 676018053, time.Local), Count:3, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.515378    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460945609, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 676036743, time.Local), Count:3, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.521013    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460955359, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 676046574, time.Local), Count:3, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.526412    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460923678, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 676981704, time.Local), Count:4, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.531893    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460945609, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 677001265, time.Local), Count:4, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.537379    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460955359, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 677013395, time.Local), Count:4, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.543069    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460923678, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 677674060, time.Local), Count:5, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.549749    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460945609, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 677692580, time.Local), Count:5, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.558804    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460955359, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 677701640, time.Local), Count:5, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.564237    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460923678, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 678095509, time.Local), Count:6, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.571757    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460945609, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 678138790, time.Local), Count:6, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.577675    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460955359, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 678151390, time.Local), Count:6, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.582931    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460923678, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 715868697, time.Local), Count:7, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.588612    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460945609, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 715930118, time.Local), Count:7, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.594694    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460955359, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 715985749, time.Local), Count:7, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2be6df" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.600776    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460923678, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 13, 122193070, time.Local), Count:8, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2b6b1e" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.606432    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 12, 460945609, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 13, 122208661, time.Local), Count:8, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e851b2bc0c9" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.613066    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e8748b8426e", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"74b8c21149c0164c4da6a5858bca8c45", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.initContainers{render-config-coredns}"}, Reason:"Pulled", Message:"Container image \"registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8\" already present on machine", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 21, 815063150, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 21, 815063150, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.618685    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e8748ba7a5f", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"ef062c82f679a1ef42be9dd35e115e2f", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.initContainers{render-config-keepalived}"}, Reason:"Pulled", Message:"Container image \"registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8\" already present on machine", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 21, 815208543, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 21, 815208543, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.624358    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e8778524727", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"ef062c82f679a1ef42be9dd35e115e2f", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.initContainers{render-config-keepalived}"}, Reason:"Created", Message:"Created container render-config-keepalived", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 22, 613686055, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 22, 613686055, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.630092    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e8778babfcf", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"74b8c21149c0164c4da6a5858bca8c45", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.initContainers{render-config-coredns}"}, Reason:"Created", Message:"Created container render-config-coredns", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 22, 620532687, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 22, 620532687, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.635427    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e877a5c13dd", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"ef062c82f679a1ef42be9dd35e115e2f", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.initContainers{render-config-keepalived}"}, Reason:"Started", Message:"Started container render-config-keepalived", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 22, 647882717, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 22, 647882717, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.640762    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e879908f46f", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"74b8c21149c0164c4da6a5858bca8c45", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.initContainers{render-config-coredns}"}, Reason:"Started", Message:"Started container render-config-coredns", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 23, 162528879, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 23, 162528879, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.646807    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e87b338e390", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"ef062c82f679a1ef42be9dd35e115e2f", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{keepalived}"}, Reason:"Pulling", Message:"Pulling image \"registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:54f65db0a061a8364acccc26cbcbeea96f018ef0adbe8de3eee225ac3b386fc4\"", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 23, 601877904, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 23, 601877904, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.652253    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e87b349c112", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"74b8c21149c0164c4da6a5858bca8c45", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{coredns}"}, Reason:"Pulling", Message:"Pulling image \"registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:fb2d31169676893fd6475bb59f5c7d0d74664da516e8cb324deed7896d995309\"", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 23, 602983186, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 23, 602983186, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.658106    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e892cfd3ec2", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"74b8c21149c0164c4da6a5858bca8c45", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{coredns}"}, Reason:"Pulled", Message:"Successfully pulled image \"registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:fb2d31169676893fd6475bb59f5c7d0d74664da516e8cb324deed7896d995309\" in 6.336s (6.336s including waiting)", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 29, 939756738, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 29, 939756738, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.663486    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e8932b43f30", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"ef062c82f679a1ef42be9dd35e115e2f", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{keepalived}"}, Reason:"Pulled", Message:"Successfully pulled image \"registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:54f65db0a061a8364acccc26cbcbeea96f018ef0adbe8de3eee225ac3b386fc4\" in 6.433s (6.433s including waiting)", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 35636016, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 35636016, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.669131    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e8939221d06", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"ef062c82f679a1ef42be9dd35e115e2f", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{keepalived}"}, Reason:"Created", Message:"Created container keepalived", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 143499526, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 143499526, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.676298    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e893b28a906", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"ef062c82f679a1ef42be9dd35e115e2f", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{keepalived}"}, Reason:"Started", Message:"Started container keepalived", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 177483014, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 177483014, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.681627    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e893b69f873", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"ef062c82f679a1ef42be9dd35e115e2f", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{keepalived-monitor}"}, Reason:"Pulled", Message:"Container image \"registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8\" already present on machine", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 181763187, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 181763187, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.687070    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e8943e8f13b", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"ef062c82f679a1ef42be9dd35e115e2f", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{keepalived-monitor}"}, Reason:"Created", Message:"Created container keepalived-monitor", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 324302139, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 324302139, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.692243    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e894600661b", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"ef062c82f679a1ef42be9dd35e115e2f", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{keepalived-monitor}"}, Reason:"Started", Message:"Started container keepalived-monitor", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 359393819, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 30, 359393819, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.698471    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e896f24f4f9", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"74b8c21149c0164c4da6a5858bca8c45", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{coredns}"}, Reason:"Created", Message:"Created container coredns", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 31, 49655545, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 31, 49655545, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.704443    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e89708f296d", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"74b8c21149c0164c4da6a5858bca8c45", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{coredns}"}, Reason:"Started", Message:"Started container coredns", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 31, 73393005, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 31, 73393005, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.709775    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e897099b73d", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"74b8c21149c0164c4da6a5858bca8c45", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{coredns-monitor}"}, Reason:"Pulled", Message:"Container image \"registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8\" already present on machine", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 31, 74084669, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 31, 74084669, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.715146    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e89799949fc", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"74b8c21149c0164c4da6a5858bca8c45", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{coredns-monitor}"}, Reason:"Created", Message:"Created container coredns-monitor", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 31, 225051644, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 31, 225051644, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:34.720298    4293 event.go:280] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916e897b2cc517", GenerateName:"", Namespace:"openshift-openstack-infra", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Pod", Namespace:"openshift-openstack-infra", Name:"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"74b8c21149c0164c4da6a5858bca8c45", APIVersion:"v1", ResourceVersion:"", FieldPath:"spec.containers{coredns-monitor}"}, Reason:"Started", Message:"Started container coredns-monitor", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 54, 31, 251494167, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 54, 31, 251494167, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "openshift-openstack-infra"' (will not retry!)
Oct 25 18:54:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:35.436593    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:36.430623    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:37.431228    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:38.430635    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:39.073372    4293 controller.go:146] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Oct 25 18:54:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:39.160737    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:39.161942    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:39.161972    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:39.161983    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:39.162014    4293 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:39.168493    4293 kubelet_node_status.go:94] "Unable to register node with API server" err="nodes is forbidden: User \"system:anonymous\" cannot create resource \"nodes\" in API group \"\" at the cluster scope" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:39.430413    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:40.431482    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:41.430643    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:42.431015    4293 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 25 18:54:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:42.507880    4293 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:43.020664    4293 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Rotating certificates
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:43.077989    4293 csr.go:261] certificate signing request csr-gzrbm is approved, waiting to be issued
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:43.084603    4293 csr.go:257] certificate signing request csr-gzrbm is issued
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:43.340584    4293 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:43.436930    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:43.452877    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:43.511410    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:43.792802    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:43.792830    4293 csi_plugin.go:295] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:43.818741    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:43.837083    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:43.893220    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:44.086003    4293 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Certificate expiration is 2023-10-26 18:14:30 +0000 UTC, rotation deadline is 2023-10-26 15:35:58.587868322 +0000 UTC
Oct 25 18:54:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:44.086028    4293 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Waiting 20h41m14.501841929s for next certificate rotation
Oct 25 18:54:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:44.155657    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:44.155671    4293 csi_plugin.go:295] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:44.258232    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:44.274157    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:44.332401    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:44.612944    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:44.613174    4293 csi_plugin.go:295] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:45.189538    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:45.204331    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:45.268594    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:45.537774    4293 nodeinfomanager.go:401] Failed to publish CSINode: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:45.537788    4293 csi_plugin.go:295] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" not found
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:46.079475    4293 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:46.168760    4293 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:46.169559    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:46.169588    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:46.169599    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:46.169632    4293 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:46.270548    4293 kubelet_node_status.go:75] "Successfully registered node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:46.291038    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:46.391625    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:46.428101    4293 certificate_manager.go:356] kubernetes.io/kubelet-serving: Rotating certificates
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:46.492111    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:46.547652    4293 log.go:194] http: TLS handshake error from 10.0.2.134:51166: no serving certificate available for the kubelet
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:46.575203    4293 csr.go:261] certificate signing request csr-vhhnj is approved, waiting to be issued
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:46.592295    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:46.595611    4293 csr.go:257] certificate signing request csr-vhhnj is issued
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:46.693217    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:46.793707    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:46.894327    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:46.995351    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:47.095963    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:47.196428    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:47.297104    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:47.397463    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:47.498104    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:47.596099    4293 certificate_manager.go:356] kubernetes.io/kubelet-serving: Certificate expiration is 2023-10-26 18:14:30 +0000 UTC, rotation deadline is 2023-10-26 12:58:55.175145933 +0000 UTC
Oct 25 18:54:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:47.596167    4293 certificate_manager.go:356] kubernetes.io/kubelet-serving: Waiting 18h4m7.57898241s for next certificate rotation
Oct 25 18:54:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:47.598198    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:47.698596    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:47.799239    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:47.899899    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:48.000388    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:48.100901    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:48.201444    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:48.302088    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:48.402757    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:48.503512    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:48.596246    4293 certificate_manager.go:356] kubernetes.io/kubelet-serving: Certificate expiration is 2023-10-26 18:14:30 +0000 UTC, rotation deadline is 2023-10-26 12:12:12.141974711 +0000 UTC
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:48.596266    4293 certificate_manager.go:356] kubernetes.io/kubelet-serving: Waiting 17h17m23.545710729s for next certificate rotation
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:48.604401    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:48.704816    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:48.805447    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:48.906485    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:49.007244    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:49.107707    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:49.208157    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:49.308499    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:49.409066    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:49.509773    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:49.610823    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:49.711213    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:49.811751    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:49.912389    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:50.013390    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:50.113927    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:50.214483    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:50.315549    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:50.416238    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:50.516793    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:50.617809    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:50.718274    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:50.818791    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:50.919396    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:51.019920    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:51.120420    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:51.221058    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:51.321573    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:51.422373    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:51.522796    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:51.623706    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:51.724295    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:51.824762    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:51.925322    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.025822    4293 kubelet_node_status.go:485] "Error getting the current node from lister" err="node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.415419    4293 apiserver.go:52] "Watching apiserver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.421874    4293 kubelet.go:2399] "SyncLoop ADD" source="api" pods=["openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m","openshift-multus/multus-additional-cni-plugins-qngxd","openshift-multus/multus-j4st5","openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj","openshift-dns/node-resolver-qngzk","openshift-image-registry/node-ca-rfcb9","openshift-machine-config-operator/machine-config-daemon-zmn8s","openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm","openshift-monitoring/node-exporter-6mp47","openshift-multus/network-metrics-daemon-ckj6x","openshift-network-diagnostics/network-check-target-gg7qk","openshift-cluster-node-tuning-operator/tuned-29pm7","openshift-ovn-kubernetes/ovnkube-node-wc25f"]
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.421900    4293 topology_manager.go:215] "Topology Admit Handler" podUID="ce1d733d-5d0c-42c3-93bf-254808356343" podNamespace="openshift-manila-csi-driver" podName="csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.421916    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.421939    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.421945    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.421951    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.421956    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.421961    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.421971    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.421977    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.421982    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422041    4293 topology_manager.go:215] "Topology Admit Handler" podUID="75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06" podNamespace="openshift-multus" podName="multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422051    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="egress-router-binary-copy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422064    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="egress-router-binary-copy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422073    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="egress-router-binary-copy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422080    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422086    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422090    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422096    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422101    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="egress-router-binary-copy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422111    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="egress-router-binary-copy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422138    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="cni-plugins"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422146    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="cni-plugins"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422151    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="cni-plugins"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422157    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422163    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422169    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422176    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422183    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="cni-plugins"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422190    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="cni-plugins"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422208    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="bond-cni-plugin"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422218    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="bond-cni-plugin"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422226    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="bond-cni-plugin"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422233    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422239    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422246    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422254    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422261    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="bond-cni-plugin"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422269    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="bond-cni-plugin"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422284    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="routeoverride-cni"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422297    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="routeoverride-cni"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422306    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="routeoverride-cni"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422310    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422314    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422319    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422325    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422332    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="routeoverride-cni"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422340    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="routeoverride-cni"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422356    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="whereabouts-cni-bincopy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422367    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="whereabouts-cni-bincopy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422376    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="whereabouts-cni-bincopy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422383    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422389    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422396    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422405    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422414    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="whereabouts-cni-bincopy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422421    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="whereabouts-cni-bincopy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422438    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="whereabouts-cni"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422449    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="whereabouts-cni"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422457    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="whereabouts-cni"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422463    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422467    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422472    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422478    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422483    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="whereabouts-cni"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422488    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="whereabouts-cni"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422500    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="kube-multus-additional-cni-plugins"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422508    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="kube-multus-additional-cni-plugins"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422514    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="kube-multus-additional-cni-plugins"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422518    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422523    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422527    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422532    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422538    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="kube-multus-additional-cni-plugins"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422546    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-additional-cni-plugins-qngxd" containerName="kube-multus-additional-cni-plugins"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422595    4293 topology_manager.go:215] "Topology Admit Handler" podUID="93d972c7-c2ab-4f1e-8b56-d8d40be93cb2" podNamespace="openshift-multus" podName="multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422603    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-multus/multus-j4st5" containerName="kube-multus"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422609    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-j4st5" containerName="kube-multus"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422614    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/multus-j4st5" containerName="kube-multus"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422619    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422785    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422790    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422796    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422800    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-j4st5" containerName="kube-multus"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422805    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/multus-j4st5" containerName="kube-multus"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422849    4293 topology_manager.go:215] "Topology Admit Handler" podUID="fdf0522a-ef9e-45e0-bee2-397d22cc2f3b" podNamespace="openshift-cluster-csi-drivers" podName="openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422859    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422873    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422880    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422884    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422888    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422892    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422898    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422903    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422908    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422918    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="node-driver-registrar"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422924    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="node-driver-registrar"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422930    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="node-driver-registrar"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422934    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422938    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422943    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422948    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422953    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="node-driver-registrar"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422959    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="node-driver-registrar"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422969    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="csi-liveness-probe"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422975    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="csi-liveness-probe"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422980    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="csi-liveness-probe"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422985    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422989    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422993    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.422998    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423003    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="csi-liveness-probe"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423008    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" containerName="csi-liveness-probe"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423056    4293 topology_manager.go:215] "Topology Admit Handler" podUID="21b6dd11-c00e-4ec7-a192-703f8a824b80" podNamespace="openshift-dns" podName="node-resolver-qngzk"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423064    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-dns/node-resolver-qngzk" containerName="dns-node-resolver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423071    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-dns/node-resolver-qngzk" containerName="dns-node-resolver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423076    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-dns/node-resolver-qngzk" containerName="dns-node-resolver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423082    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423087    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423091    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423097    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423103    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-dns/node-resolver-qngzk" containerName="dns-node-resolver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423108    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-dns/node-resolver-qngzk" containerName="dns-node-resolver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423177    4293 topology_manager.go:215] "Topology Admit Handler" podUID="9954521a-e814-4b76-b21a-3b08ad936c10" podNamespace="openshift-image-registry" podName="node-ca-rfcb9"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423189    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-image-registry/node-ca-rfcb9" containerName="node-ca"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423198    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-image-registry/node-ca-rfcb9" containerName="node-ca"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423205    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-image-registry/node-ca-rfcb9" containerName="node-ca"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423211    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423216    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423222    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423230    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423236    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-image-registry/node-ca-rfcb9" containerName="node-ca"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423243    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-image-registry/node-ca-rfcb9" containerName="node-ca"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423308    4293 topology_manager.go:215] "Topology Admit Handler" podUID="aab63cd9-e2d8-4dff-8d13-22cb7a673703" podNamespace="openshift-machine-config-operator" podName="machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423323    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" containerName="machine-config-daemon"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423335    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" containerName="machine-config-daemon"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423343    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" containerName="machine-config-daemon"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423349    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423357    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423363    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423371    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423378    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" containerName="machine-config-daemon"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423385    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" containerName="machine-config-daemon"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423401    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423415    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423424    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423430    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423436    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423442    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423452    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423459    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423466    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423548    4293 topology_manager.go:215] "Topology Admit Handler" podUID="60df7285-d18c-4337-9e6b-27c667f60eee" podNamespace="openshift-manila-csi-driver" podName="openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423564    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423573    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423580    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423587    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423592    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423598    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423605    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423612    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423619    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" containerName="csi-driver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423633    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" containerName="csi-node-driver-registrar"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423643    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" containerName="csi-node-driver-registrar"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423650    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" containerName="csi-node-driver-registrar"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423656    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423663    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423669    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423676    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423683    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" containerName="csi-node-driver-registrar"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423690    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" containerName="csi-node-driver-registrar"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423804    4293 topology_manager.go:215] "Topology Admit Handler" podUID="ce8d94be-2c75-42a5-a8e1-60275cefdcbd" podNamespace="openshift-monitoring" podName="node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423834    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-monitoring/node-exporter-6mp47" containerName="init-textfile"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423852    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-monitoring/node-exporter-6mp47" containerName="init-textfile"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423861    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-monitoring/node-exporter-6mp47" containerName="init-textfile"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423870    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423876    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423882    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423896    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423903    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-monitoring/node-exporter-6mp47" containerName="init-textfile"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423914    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-monitoring/node-exporter-6mp47" containerName="init-textfile"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423935    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-monitoring/node-exporter-6mp47" containerName="node-exporter"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423945    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-monitoring/node-exporter-6mp47" containerName="node-exporter"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423953    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-monitoring/node-exporter-6mp47" containerName="node-exporter"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423959    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423964    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423970    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423978    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423986    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-monitoring/node-exporter-6mp47" containerName="node-exporter"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.423993    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-monitoring/node-exporter-6mp47" containerName="node-exporter"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424009    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-monitoring/node-exporter-6mp47" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424021    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-monitoring/node-exporter-6mp47" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424028    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-monitoring/node-exporter-6mp47" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424035    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424041    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424048    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424055    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424062    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-monitoring/node-exporter-6mp47" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424069    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-monitoring/node-exporter-6mp47" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424714    4293 topology_manager.go:215] "Topology Admit Handler" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8" podNamespace="openshift-multus" podName="network-metrics-daemon-ckj6x"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424753    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-multus/network-metrics-daemon-ckj6x" containerName="network-metrics-daemon"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424771    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/network-metrics-daemon-ckj6x" containerName="network-metrics-daemon"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424781    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/network-metrics-daemon-ckj6x" containerName="network-metrics-daemon"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424791    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424799    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424805    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424820    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424828    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/network-metrics-daemon-ckj6x" containerName="network-metrics-daemon"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424837    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/network-metrics-daemon-ckj6x" containerName="network-metrics-daemon"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424858    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-multus/network-metrics-daemon-ckj6x" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424868    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/network-metrics-daemon-ckj6x" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424878    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-multus/network-metrics-daemon-ckj6x" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424884    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424889    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424896    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424904    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424911    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/network-metrics-daemon-ckj6x" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.424919    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-multus/network-metrics-daemon-ckj6x" containerName="kube-rbac-proxy"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425069    4293 topology_manager.go:215] "Topology Admit Handler" podUID="2f285036-37bd-4419-a167-e41819a56c33" podNamespace="openshift-network-diagnostics" podName="network-check-target-gg7qk"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425080    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-network-diagnostics/network-check-target-gg7qk" containerName="network-check-target-container"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425089    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-network-diagnostics/network-check-target-gg7qk" containerName="network-check-target-container"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425097    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-network-diagnostics/network-check-target-gg7qk" containerName="network-check-target-container"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425103    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425109    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425131    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425140    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425147    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-network-diagnostics/network-check-target-gg7qk" containerName="network-check-target-container"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425155    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-network-diagnostics/network-check-target-gg7qk" containerName="network-check-target-container"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.425231    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425248    4293 topology_manager.go:215] "Topology Admit Handler" podUID="39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3" podNamespace="openshift-cluster-node-tuning-operator" podName="tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425262    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-cluster-node-tuning-operator/tuned-29pm7" containerName="tuned"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425272    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-cluster-node-tuning-operator/tuned-29pm7" containerName="tuned"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425280    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-cluster-node-tuning-operator/tuned-29pm7" containerName="tuned"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425286    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425291    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425297    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425304    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425311    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-cluster-node-tuning-operator/tuned-29pm7" containerName="tuned"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425317    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-cluster-node-tuning-operator/tuned-29pm7" containerName="tuned"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.425408    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425419    4293 topology_manager.go:215] "Topology Admit Handler" podUID="6a75dddb-d3b3-408f-9cab-46b51953ccc8" podNamespace="openshift-ovn-kubernetes" podName="ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425433    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kubecfg-setup"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425451    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kubecfg-setup"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425459    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kubecfg-setup"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425465    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425471    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425483    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425490    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425497    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kubecfg-setup"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425505    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kubecfg-setup"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425524    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovn-controller"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425536    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovn-controller"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425542    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovn-controller"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425548    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425554    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425560    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425566    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425572    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovn-controller"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425578    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovn-controller"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425594    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovn-acl-logging"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425606    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovn-acl-logging"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425613    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovn-acl-logging"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425619    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425625    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425630    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425637    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425643    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovn-acl-logging"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425658    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovn-acl-logging"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425676    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kube-rbac-proxy-node"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425688    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kube-rbac-proxy-node"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425695    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kube-rbac-proxy-node"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425701    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425707    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425712    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425719    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425725    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kube-rbac-proxy-node"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425732    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kube-rbac-proxy-node"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425751    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kube-rbac-proxy-ovn-metrics"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425767    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kube-rbac-proxy-ovn-metrics"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425775    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kube-rbac-proxy-ovn-metrics"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425781    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425786    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425792    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425799    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425806    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kube-rbac-proxy-ovn-metrics"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425815    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="kube-rbac-proxy-ovn-metrics"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425835    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="northd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425848    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="northd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425857    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="northd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425864    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425876    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425882    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425894    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425900    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="northd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425909    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="northd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425927    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="nbdb"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425940    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="nbdb"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425947    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="nbdb"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425953    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425959    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425964    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425971    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425977    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="nbdb"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.425983    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="nbdb"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426003    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="sbdb"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426014    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="sbdb"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426023    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="sbdb"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426030    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426036    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426041    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426048    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426054    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="sbdb"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426061    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="sbdb"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426077    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovnkube-controller"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426093    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovnkube-controller"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426100    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovnkube-controller"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426107    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426113    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426134    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426142    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426150    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovnkube-controller"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.426159    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" containerName="ovnkube-controller"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508473    4293 desired_state_of_world_populator.go:159] "Finished populating initial desired state of world"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508590    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-modprobe-d\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508617    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-cinderplugin\" (UniqueName: \"kubernetes.io/secret/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-secret-cinderplugin\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508636    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pods-mount-dir\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-pods-mount-dir\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508654    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fwd-plugin-dir\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-fwd-plugin-dir\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508780    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacert\" (UniqueName: \"kubernetes.io/configmap/60df7285-d18c-4337-9e6b-27c667f60eee-cacert\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508820    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-cni-bin\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508840    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21b6dd11-c00e-4ec7-a192-703f8a824b80-hosts-file\") pod \"node-resolver-qngzk\" (UID: \"21b6dd11-c00e-4ec7-a192-703f8a824b80\") " pod="openshift-dns/node-resolver-qngzk"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508860    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-host\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508879    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aab63cd9-e2d8-4dff-8d13-22cb7a673703-proxy-tls\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508929    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tzhz\" (UniqueName: \"kubernetes.io/projected/aab63cd9-e2d8-4dff-8d13-22cb7a673703-kube-api-access-8tzhz\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.508976    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-sys\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509007    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-conf-dir\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509042    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzd52\" (UniqueName: \"kubernetes.io/projected/9954521a-e814-4b76-b21a-3b08ad936c10-kube-api-access-jzd52\") pod \"node-ca-rfcb9\" (UID: \"9954521a-e814-4b76-b21a-3b08ad936c10\") " pod="openshift-image-registry/node-ca-rfcb9"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509178    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-var-lib-cni-bin\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509245    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jgbw\" (UniqueName: \"kubernetes.io/projected/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-kube-api-access-8jgbw\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509293    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-log-socket\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509334    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rqmd\" (UniqueName: \"kubernetes.io/projected/6a75dddb-d3b3-408f-9cab-46b51953ccc8-kube-api-access-6rqmd\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509366    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-os-release\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509398    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-run\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509423    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacert\" (UniqueName: \"kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-cacert\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509450    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-sys-fs\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509478    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-etc-selinux\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509506    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-system-cni-dir\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509542    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-var-lib-kubelet\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509591    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5smgg\" (UniqueName: \"kubernetes.io/projected/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-kube-api-access-5smgg\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509640    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-tuned-profiles-data\" (UniqueName: \"kubernetes.io/configmap/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-var-lib-tuned-profiles-data\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509671    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-non-standard-root-system-trust-ca-bundle\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509691    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-systemd-units\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509708    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-var-lib-kubelet\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509726    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-os-release\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509760    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-cert\" (UniqueName: \"kubernetes.io/secret/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-cert\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509781    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-etc-kubernetes\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509807    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-textfile\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509831    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509870    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4n5q\" (UniqueName: \"kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q\") pod \"network-check-target-gg7qk\" (UID: \"2f285036-37bd-4419-a167-e41819a56c33\") " pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509944    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-run-multus-certs\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.509993    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"mcd-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/aab63cd9-e2d8-4dff-8d13-22cb7a673703-mcd-auth-proxy-config\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510025    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-etc-selinux\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510072    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-cni-netd\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510128    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-run-netns\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510149    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-hostroot\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510180    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510207    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-dir\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-plugin-dir\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510242    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-run-openvswitch\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510284    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-run-k8s-cni-cncf-io\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510326    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-cnibin\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510360    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-run-ovn\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510393    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovnkube-config\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510425    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-node-metrics-cert\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510458    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-root\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510501    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkjd6\" (UniqueName: \"kubernetes.io/projected/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-kube-api-access-vkjd6\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510536    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-etc-openvswitch\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510563    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-systemd\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510586    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ntwx\" (UniqueName: \"kubernetes.io/projected/ce1d733d-5d0c-42c3-93bf-254808356343-kube-api-access-8ntwx\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510669    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-sys-fs\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510719    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60df7285-d18c-4337-9e6b-27c667f60eee-non-standard-root-system-trust-ca-bundle\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510773    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-sysctl-d\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510808    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.510837    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-kubelet\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511140    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511235    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-var-lib-cni-multus\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511293    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-sys\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511321    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9954521a-e814-4b76-b21a-3b08ad936c10-serviceca\") pod \"node-ca-rfcb9\" (UID: \"9954521a-e814-4b76-b21a-3b08ad936c10\") " pod="openshift-image-registry/node-ca-rfcb9"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511347    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-env-overrides\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511396    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-daemon-config\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511436    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511472    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-sysconfig\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511503    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-registration-dir\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511559    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-wtmp\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511610    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-run-netns\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511645    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-lib-modules\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511682    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-registration-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511719    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-etc-selinux\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511930    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8ncd\" (UniqueName: \"kubernetes.io/projected/60df7285-d18c-4337-9e6b-27c667f60eee-kube-api-access-h8ncd\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.511976    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-tls\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512030    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pods-probe-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-pods-probe-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512097    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-run-ovn-kubernetes\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512165    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-system-cni-dir\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512269    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/aab63cd9-e2d8-4dff-8d13-22cb7a673703-rootfs\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512329    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9954521a-e814-4b76-b21a-3b08ad936c10-host\") pod \"node-ca-rfcb9\" (UID: \"9954521a-e814-4b76-b21a-3b08ad936c10\") " pod="openshift-image-registry/node-ca-rfcb9"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512367    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-cni-dir\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512393    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-sysctl-conf\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512439    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-var-lib-openvswitch\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512478    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf5gb\" (UniqueName: \"kubernetes.io/projected/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-kube-api-access-xf5gb\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512533    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-socket-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512570    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-kubelet-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512632    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-sys-fs\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512715    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-dir\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-plugin-dir\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512749    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-socket-dir-parent\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512781    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvsgx\" (UniqueName: \"kubernetes.io/projected/38bac755-4b7a-447a-bea0-6c1be8066cd8-kube-api-access-kvsgx\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.512810    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6vvr\" (UniqueName: \"kubernetes.io/projected/21b6dd11-c00e-4ec7-a192-703f8a824b80-kube-api-access-g6vvr\") pod \"node-resolver-qngzk\" (UID: \"21b6dd11-c00e-4ec7-a192-703f8a824b80\") " pod="openshift-dns/node-resolver-qngzk"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.513098    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-ca\" (UniqueName: \"kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-ca\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.513170    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-cnibin\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.513203    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-cinderplugin\" (UniqueName: \"kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-config-cinderplugin\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.513233    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-cni-binary-copy\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.513401    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-metrics-client-ca\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.513445    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-slash\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.513495    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zszm\" (UniqueName: \"kubernetes.io/projected/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-kube-api-access-9zszm\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.513539    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-node-log\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.513577    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-cni-binary-copy\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.513605    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-kubernetes\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614351    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-systemd-units\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614384    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-var-lib-kubelet\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614402    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-os-release\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614425    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"ovn-cert\" (UniqueName: \"kubernetes.io/secret/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-cert\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614447    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-etc-kubernetes\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614466    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-textfile\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614467    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-var-lib-kubelet\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614486    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614505    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-systemd-units\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614509    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-n4n5q\" (UniqueName: \"kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q\") pod \"network-check-target-gg7qk\" (UID: \"2f285036-37bd-4419-a167-e41819a56c33\") " pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614528    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-cni-netd\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614549    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-run-netns\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614557    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-os-release\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614565    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-hostroot\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614585    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-run-multus-certs\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614605    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"mcd-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/aab63cd9-e2d8-4dff-8d13-22cb7a673703-mcd-auth-proxy-config\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614624    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-etc-selinux\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614641    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-run-openvswitch\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614659    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-run-k8s-cni-cncf-io\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614685    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614694    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-run-netns\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614712    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"plugin-dir\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-plugin-dir\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614725    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-cni-netd\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614530    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-etc-kubernetes\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614742    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-run-ovn\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614769    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovnkube-config\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615069    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-node-metrics-cert\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615070    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-run-ovn\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615089    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-cnibin\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614794    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-run-multus-certs\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614804    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-etc-selinux\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615110    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-systemd\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615165    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntwx\" (UniqueName: \"kubernetes.io/projected/ce1d733d-5d0c-42c3-93bf-254808356343-kube-api-access-8ntwx\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615197    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-sys-fs\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615230    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60df7285-d18c-4337-9e6b-27c667f60eee-non-standard-root-system-trust-ca-bundle\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615256    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-root\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615283    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-vkjd6\" (UniqueName: \"kubernetes.io/projected/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-kube-api-access-vkjd6\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615309    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-etc-openvswitch\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615335    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-kubelet\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615361    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovnkube-config\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615365    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615421    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-sysctl-d\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615449    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615478    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-var-lib-cni-multus\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615503    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-sys\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615529    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-env-overrides\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615557    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-daemon-config\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615613    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615643    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9954521a-e814-4b76-b21a-3b08ad936c10-serviceca\") pod \"node-ca-rfcb9\" (UID: \"9954521a-e814-4b76-b21a-3b08ad936c10\") " pod="openshift-image-registry/node-ca-rfcb9"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615672    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-registration-dir\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615699    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-wtmp\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615727    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-sysconfig\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615755    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-etc-selinux\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615787    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-h8ncd\" (UniqueName: \"kubernetes.io/projected/60df7285-d18c-4337-9e6b-27c667f60eee-kube-api-access-h8ncd\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615817    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-tls\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615833    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"mcd-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/aab63cd9-e2d8-4dff-8d13-22cb7a673703-mcd-auth-proxy-config\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615845    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-run-netns\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615872    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-lib-modules\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615900    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-registration-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615926    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"pods-probe-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-pods-probe-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615965    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-run-ovn-kubernetes\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615993    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-system-cni-dir\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616019    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-root\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616021    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/aab63cd9-e2d8-4dff-8d13-22cb7a673703-rootfs\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616056    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/aab63cd9-e2d8-4dff-8d13-22cb7a673703-rootfs\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616075    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-cni-dir\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616080    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-cnibin\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616105    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-sysctl-conf\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616144    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-var-lib-cni-multus\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616149    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9954521a-e814-4b76-b21a-3b08ad936c10-host\") pod \"node-ca-rfcb9\" (UID: \"9954521a-e814-4b76-b21a-3b08ad936c10\") " pod="openshift-image-registry/node-ca-rfcb9"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616184    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9954521a-e814-4b76-b21a-3b08ad936c10-host\") pod \"node-ca-rfcb9\" (UID: \"9954521a-e814-4b76-b21a-3b08ad936c10\") " pod="openshift-image-registry/node-ca-rfcb9"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614952    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-textfile\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615031    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-run-k8s-cni-cncf-io\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616215    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-var-lib-openvswitch\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615032    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"plugin-dir\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-plugin-dir\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615169    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-systemd\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.615393    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614834    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-hostroot\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616273    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-sysctl-d\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616277    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-sys-fs\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616293    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616354    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-sys\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616383    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616439    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-cni-dir\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.614996    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-run-openvswitch\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616488    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-etc-openvswitch\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616571    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9954521a-e814-4b76-b21a-3b08ad936c10-serviceca\") pod \"node-ca-rfcb9\" (UID: \"9954521a-e814-4b76-b21a-3b08ad936c10\") " pod="openshift-image-registry/node-ca-rfcb9"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616633    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-registration-dir\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616671    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-env-overrides\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616723    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-sysconfig\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616772    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-wtmp\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616774    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-etc-selinux\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616820    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-lib-modules\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616877    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-registration-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616892    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60df7285-d18c-4337-9e6b-27c667f60eee-non-standard-root-system-trust-ca-bundle\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616921    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"pods-probe-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-pods-probe-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616963    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-run-ovn-kubernetes\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617020    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-sysctl-conf\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617028    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-system-cni-dir\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616185    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-var-lib-openvswitch\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.616673    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-run-netns\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617072    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-xf5gb\" (UniqueName: \"kubernetes.io/projected/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-kube-api-access-xf5gb\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617079    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-kubelet\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617104    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-socket-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617154    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"plugin-dir\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-plugin-dir\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617185    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-socket-dir-parent\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617214    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-kvsgx\" (UniqueName: \"kubernetes.io/projected/38bac755-4b7a-447a-bea0-6c1be8066cd8-kube-api-access-kvsgx\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617235    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-daemon-config\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617242    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-kubelet-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617272    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-sys-fs\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617280    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"plugin-dir\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-plugin-dir\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617298    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"ovn-ca\" (UniqueName: \"kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-ca\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617323    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-cnibin\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617345    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-socket-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617349    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"config-cinderplugin\" (UniqueName: \"kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-config-cinderplugin\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617377    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-g6vvr\" (UniqueName: \"kubernetes.io/projected/21b6dd11-c00e-4ec7-a192-703f8a824b80-kube-api-access-g6vvr\") pod \"node-resolver-qngzk\" (UID: \"21b6dd11-c00e-4ec7-a192-703f8a824b80\") " pod="openshift-dns/node-resolver-qngzk"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617401    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-socket-dir-parent\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617404    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-metrics-client-ca\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617428    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-slash\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617455    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-9zszm\" (UniqueName: \"kubernetes.io/projected/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-kube-api-access-9zszm\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617480    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-cni-binary-copy\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617507    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-node-log\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617557    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-cni-binary-copy\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617586    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-kubernetes\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617618    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"fwd-plugin-dir\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-fwd-plugin-dir\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617645    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-cnibin\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617649    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"cacert\" (UniqueName: \"kubernetes.io/configmap/60df7285-d18c-4337-9e6b-27c667f60eee-cacert\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617695    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-cni-bin\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617722    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-modprobe-d\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617748    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"secret-cinderplugin\" (UniqueName: \"kubernetes.io/secret/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-secret-cinderplugin\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617774    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"pods-mount-dir\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-pods-mount-dir\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617802    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21b6dd11-c00e-4ec7-a192-703f8a824b80-hosts-file\") pod \"node-resolver-qngzk\" (UID: \"21b6dd11-c00e-4ec7-a192-703f8a824b80\") " pod="openshift-dns/node-resolver-qngzk"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617824    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-node-log\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617828    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-host\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617862    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-host\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617379    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-kubelet-dir\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617884    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aab63cd9-e2d8-4dff-8d13-22cb7a673703-proxy-tls\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617913    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-cni-bin\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617946    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"ovn-ca\" (UniqueName: \"kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-ca\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617956    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-8tzhz\" (UniqueName: \"kubernetes.io/projected/aab63cd9-e2d8-4dff-8d13-22cb7a673703-kube-api-access-8tzhz\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617993    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-sys\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.617998    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-modprobe-d\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618023    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-conf-dir\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618040    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-etc-kubernetes\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618053    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-jzd52\" (UniqueName: \"kubernetes.io/projected/9954521a-e814-4b76-b21a-3b08ad936c10-kube-api-access-jzd52\") pod \"node-ca-rfcb9\" (UID: \"9954521a-e814-4b76-b21a-3b08ad936c10\") " pod="openshift-image-registry/node-ca-rfcb9"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618066    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-metrics-client-ca\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618081    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-log-socket\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618110    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-6rqmd\" (UniqueName: \"kubernetes.io/projected/6a75dddb-d3b3-408f-9cab-46b51953ccc8-kube-api-access-6rqmd\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618146    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"fwd-plugin-dir\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-fwd-plugin-dir\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618156    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-os-release\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618168    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-host-slash\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618193    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-var-lib-cni-bin\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618225    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-8jgbw\" (UniqueName: \"kubernetes.io/projected/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-kube-api-access-8jgbw\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618254    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-etc-selinux\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618281    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-system-cni-dir\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618307    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-var-lib-kubelet\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618332    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-run\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618358    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"cacert\" (UniqueName: \"kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-cacert\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618386    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-sys-fs\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618413    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"var-lib-tuned-profiles-data\" (UniqueName: \"kubernetes.io/configmap/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-var-lib-tuned-profiles-data\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618441    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-non-standard-root-system-trust-ca-bundle\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618470    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-5smgg\" (UniqueName: \"kubernetes.io/projected/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-kube-api-access-5smgg\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618508    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-cni-binary-copy\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618523    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-cni-binary-copy\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618601    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21b6dd11-c00e-4ec7-a192-703f8a824b80-hosts-file\") pod \"node-resolver-qngzk\" (UID: \"21b6dd11-c00e-4ec7-a192-703f8a824b80\") " pod="openshift-dns/node-resolver-qngzk"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618650    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"pods-mount-dir\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-pods-mount-dir\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618665    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-sys\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618693    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/60df7285-d18c-4337-9e6b-27c667f60eee-etc-selinux\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618702    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-var-lib-kubelet\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618708    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-run\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618737    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-conf-dir\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618754    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-system-cni-dir\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.618893    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a75dddb-d3b3-408f-9cab-46b51953ccc8-log-socket\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.619065    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"cacert\" (UniqueName: \"kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-cacert\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.619071    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-os-release\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.619060    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"var-lib-tuned-profiles-data\" (UniqueName: \"kubernetes.io/configmap/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-var-lib-tuned-profiles-data\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.619136    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-host-var-lib-cni-bin\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.619164    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ce1d733d-5d0c-42c3-93bf-254808356343-sys-fs\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.619308    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"non-standard-root-system-trust-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-non-standard-root-system-trust-ca-bundle\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.619406    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"cacert\" (UniqueName: \"kubernetes.io/configmap/60df7285-d18c-4337-9e6b-27c667f60eee-cacert\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.619442    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-sys-fs\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.619545    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"config-cinderplugin\" (UniqueName: \"kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-config-cinderplugin\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.620398    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-node-metrics-cert\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.620517    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"ovn-cert\" (UniqueName: \"kubernetes.io/secret/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-cert\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.620578    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"secret-cinderplugin\" (UniqueName: \"kubernetes.io/secret/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-secret-cinderplugin\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.623706    4293 secret.go:194] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.623794    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs podName:38bac755-4b7a-447a-bea0-6c1be8066cd8 nodeName:}" failed. No retries permitted until 2023-10-25 18:54:53.123755248 +0000 UTC m=+41.292485313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs") pod "network-metrics-daemon-ckj6x" (UID: "38bac755-4b7a-447a-bea0-6c1be8066cd8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.624156    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.625048    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-tls\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.625569    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aab63cd9-e2d8-4dff-8d13-22cb7a673703-proxy-tls\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.678970    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzd52\" (UniqueName: \"kubernetes.io/projected/9954521a-e814-4b76-b21a-3b08ad936c10-kube-api-access-jzd52\") pod \"node-ca-rfcb9\" (UID: \"9954521a-e814-4b76-b21a-3b08ad936c10\") " pod="openshift-image-registry/node-ca-rfcb9"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.679685    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf5gb\" (UniqueName: \"kubernetes.io/projected/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-kube-api-access-xf5gb\") pod \"multus-j4st5\" (UID: \"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2\") " pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.698781    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ntwx\" (UniqueName: \"kubernetes.io/projected/ce1d733d-5d0c-42c3-93bf-254808356343-kube-api-access-8ntwx\") pod \"csi-nodeplugin-nfsplugin-j9n8m\" (UID: \"ce1d733d-5d0c-42c3-93bf-254808356343\") " pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.703922    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkjd6\" (UniqueName: \"kubernetes.io/projected/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-kube-api-access-vkjd6\") pod \"node-exporter-6mp47\" (UID: \"ce8d94be-2c75-42a5-a8e1-60275cefdcbd\") " pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.704337    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zszm\" (UniqueName: \"kubernetes.io/projected/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-kube-api-access-9zszm\") pod \"tuned-29pm7\" (UID: \"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3\") " pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.709790    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8ncd\" (UniqueName: \"kubernetes.io/projected/60df7285-d18c-4337-9e6b-27c667f60eee-kube-api-access-h8ncd\") pod \"openstack-manila-csi-nodeplugin-nclxm\" (UID: \"60df7285-d18c-4337-9e6b-27c667f60eee\") " pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.710183    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rqmd\" (UniqueName: \"kubernetes.io/projected/6a75dddb-d3b3-408f-9cab-46b51953ccc8-kube-api-access-6rqmd\") pod \"ovnkube-node-wc25f\" (UID: \"6a75dddb-d3b3-408f-9cab-46b51953ccc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.718342    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.718370    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.718382    4293 projected.go:198] Error preparing data for projected volume kube-api-access-n4n5q for pod openshift-network-diagnostics/network-check-target-gg7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.718447    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q podName:2f285036-37bd-4419-a167-e41819a56c33 nodeName:}" failed. No retries permitted until 2023-10-25 18:54:53.218429582 +0000 UTC m=+41.387159697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n4n5q" (UniqueName: "kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q") pod "network-check-target-gg7qk" (UID: "2f285036-37bd-4419-a167-e41819a56c33") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.721891    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6vvr\" (UniqueName: \"kubernetes.io/projected/21b6dd11-c00e-4ec7-a192-703f8a824b80-kube-api-access-g6vvr\") pod \"node-resolver-qngzk\" (UID: \"21b6dd11-c00e-4ec7-a192-703f8a824b80\") " pod="openshift-dns/node-resolver-qngzk"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.723132    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jgbw\" (UniqueName: \"kubernetes.io/projected/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-kube-api-access-8jgbw\") pod \"multus-additional-cni-plugins-qngxd\" (UID: \"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06\") " pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.730306    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvsgx\" (UniqueName: \"kubernetes.io/projected/38bac755-4b7a-447a-bea0-6c1be8066cd8-kube-api-access-kvsgx\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.731318    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-5smgg\" (UniqueName: \"kubernetes.io/projected/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-kube-api-access-5smgg\") pod \"openstack-cinder-csi-driver-node-p7lgj\" (UID: \"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b\") " pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.731562    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tzhz\" (UniqueName: \"kubernetes.io/projected/aab63cd9-e2d8-4dff-8d13-22cb7a673703-kube-api-access-8tzhz\") pod \"machine-config-daemon-zmn8s\" (UID: \"aab63cd9-e2d8-4dff-8d13-22cb7a673703\") " pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.754339    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.762403    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qngxd"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.763355    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j4st5"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.763987    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rfcb9"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.764909    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/machine-config-daemon-zmn8s"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.766293    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6mp47"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.766534    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.770245    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:52.770817    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1d733d_5d0c_42c3_93bf_254808356343.slice/crio-36edd484a40ed07995e9fa3d0c8ed9e25a9dc0d313e1160f30d41fe5d37c7db2 WatchSource:0}: Error finding container 36edd484a40ed07995e9fa3d0c8ed9e25a9dc0d313e1160f30d41fe5d37c7db2: Status 404 returned error can't find the container with id 36edd484a40ed07995e9fa3d0c8ed9e25a9dc0d313e1160f30d41fe5d37c7db2
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.776139    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qngzk"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.781079    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-29pm7"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:52.783877    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:52.784868    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d972c7_c2ab_4f1e_8b56_d8d40be93cb2.slice/crio-c17087d9d7aeae01a96e0d0a54bd993a857d3290aa26e0fbd3f2effd501619c4 WatchSource:0}: Error finding container c17087d9d7aeae01a96e0d0a54bd993a857d3290aa26e0fbd3f2effd501619c4: Status 404 returned error can't find the container with id c17087d9d7aeae01a96e0d0a54bd993a857d3290aa26e0fbd3f2effd501619c4
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:52.794382    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab63cd9_e2d8_4dff_8d13_22cb7a673703.slice/crio-a004f9f448933efced89461c741f879facaf81b713e0f8fc67549d3866d2d30a WatchSource:0}: Error finding container a004f9f448933efced89461c741f879facaf81b713e0f8fc67549d3866d2d30a: Status 404 returned error can't find the container with id a004f9f448933efced89461c741f879facaf81b713e0f8fc67549d3866d2d30a
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:52.795072    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9954521a_e814_4b76_b21a_3b08ad936c10.slice/crio-643df05148165c529d31e747ec90684bb8063416038b053118722e91c2cd5fae WatchSource:0}: Error finding container 643df05148165c529d31e747ec90684bb8063416038b053118722e91c2cd5fae: Status 404 returned error can't find the container with id 643df05148165c529d31e747ec90684bb8063416038b053118722e91c2cd5fae
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:52.796558    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce8d94be_2c75_42a5_a8e1_60275cefdcbd.slice/crio-98a3db0785877704f99c80581ed812e233d83013582159ac9ca292d61f8a3972 WatchSource:0}: Error finding container 98a3db0785877704f99c80581ed812e233d83013582159ac9ca292d61f8a3972: Status 404 returned error can't find the container with id 98a3db0785877704f99c80581ed812e233d83013582159ac9ca292d61f8a3972
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:52.799900    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60df7285_d18c_4337_9e6b_27c667f60eee.slice/crio-f03e5eb4d48dd84ae23a7812bdb5e73a8c61ccad2f449e7c77bf5740d2ed9d87 WatchSource:0}: Error finding container f03e5eb4d48dd84ae23a7812bdb5e73a8c61ccad2f449e7c77bf5740d2ed9d87: Status 404 returned error can't find the container with id f03e5eb4d48dd84ae23a7812bdb5e73a8c61ccad2f449e7c77bf5740d2ed9d87
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.803581    4293 kuberuntime_manager.go:1256] container &Container{Name:machine-config-daemon,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:10851816140a64d6b1b12fec976948886faab5cddf5048599865b27506b1a357,Command:[/usr/bin/machine-config-daemon],Args:[start --payload-version=4.15.0-0.ci.test-2023-10-25-180434-ci-op-nndncs4w-latest],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:NODE_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:spec.nodeName,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{20 -3} {<nil>} 20m DecimalSI},memory: {{52428800 0} {<nil>} 50Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:rootfs,ReadOnly:false,MountPath:/rootfs,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:kube-api-access-8tzhz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:*true,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:*false,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:FallbackToLogsOnError,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod machine-config-daemon-zmn8s_openshift-machine-config-operator(aab63cd9-e2d8-4dff-8d13-22cb7a673703): CreateContainerConfigError: services have not yet been read at least once, cannot construct envvars
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:52.804494    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ff7e3b_ff7a_4e89_bbf1_c0cedc0d1cd3.slice/crio-9de7d039c2a0cec9b42b46da0710b28b59ddb9e4da3f1b54f3f8d835f2ae9642 WatchSource:0}: Error finding container 9de7d039c2a0cec9b42b46da0710b28b59ddb9e4da3f1b54f3f8d835f2ae9642: Status 404 returned error can't find the container with id 9de7d039c2a0cec9b42b46da0710b28b59ddb9e4da3f1b54f3f8d835f2ae9642
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:52.810245    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a75dddb_d3b3_408f_9cab_46b51953ccc8.slice/crio-b27c7e05f2f68c88e5b205b6d5bce3305d4efbda9bba0f633ad04ea10eecbe1d WatchSource:0}: Error finding container b27c7e05f2f68c88e5b205b6d5bce3305d4efbda9bba0f633ad04ea10eecbe1d: Status 404 returned error can't find the container with id b27c7e05f2f68c88e5b205b6d5bce3305d4efbda9bba0f633ad04ea10eecbe1d
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:54:52.815729    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b6dd11_c00e_4ec7_a192_703f8a824b80.slice/crio-878a7752aba6f8cbfdc58b21858ba3e0abc8499187acbfbdf74da5460e268f9b WatchSource:0}: Error finding container 878a7752aba6f8cbfdc58b21858ba3e0abc8499187acbfbdf74da5460e268f9b: Status 404 returned error can't find the container with id 878a7752aba6f8cbfdc58b21858ba3e0abc8499187acbfbdf74da5460e268f9b
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.819653    4293 kuberuntime_manager.go:1256] container &Container{Name:dns-node-resolver,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:6cfcf1895bb82eef1d4bcad458d207cb333e676f8f8155d7c7515d07b728ffed,Command:[/bin/bash -c #!/bin/bash
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: set -uo pipefail
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: trap 'jobs -p | xargs kill || true; wait; exit 0' TERM
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: OPENSHIFT_MARKER="openshift-generated-node-resolver"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: HOSTS_FILE="/etc/hosts"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: TEMP_FILE="/etc/hosts.tmp"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: IFS=', ' read -r -a services <<< "${SERVICES}"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: # Make a temporary file with the old hosts file's attributes.
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: if ! cp -f --attributes-only "${HOSTS_FILE}" "${TEMP_FILE}"; then
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   echo "Failed to preserve hosts file. Exiting."
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   exit 1
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: fi
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: while true; do
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   declare -A svc_ips
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   for svc in "${services[@]}"; do
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     # Fetch service IP from cluster dns if present. We make several tries
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     # to do it: IPv4, IPv6, IPv4 over TCP and IPv6 over TCP. The two last ones
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     # are for deployments with Kuryr on older OpenStack (OSP13) - those do not
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     # support UDP loadbalancers and require reaching DNS through TCP.
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     cmds=('dig -t A @"${NAMESERVER}" +short "${svc}.${CLUSTER_DOMAIN}"|grep -v "^;"'
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:           'dig -t AAAA @"${NAMESERVER}" +short "${svc}.${CLUSTER_DOMAIN}"|grep -v "^;"'
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:           'dig -t A +tcp +retry=0 @"${NAMESERVER}" +short "${svc}.${CLUSTER_DOMAIN}"|grep -v "^;"'
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:           'dig -t AAAA +tcp +retry=0 @"${NAMESERVER}" +short "${svc}.${CLUSTER_DOMAIN}"|grep -v "^;"')
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     for i in ${!cmds[*]}
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     do
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:       ips=($(eval "${cmds[i]}"))
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:       if [[ "$?" -eq 0 && "${#ips[@]}" -ne 0 ]]; then
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:         svc_ips["${svc}"]="${ips[@]}"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:         break
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:       fi
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     done
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   done
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   # Update /etc/hosts only if we get valid service IPs
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   # We will not update /etc/hosts when there is coredns service outage or api unavailability
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   # Stale entries could exist in /etc/hosts if the service is deleted
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   if [[ -n "${svc_ips[*]-}" ]]; then
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     # Build a new hosts file from /etc/hosts with our custom entries filtered out
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     if ! sed --silent "/# ${OPENSHIFT_MARKER}/d; w ${TEMP_FILE}" "${HOSTS_FILE}"; then
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:       # Only continue rebuilding the hosts entries if its original content is preserved
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:       sleep 60 & wait
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:       continue
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     fi
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     # Append resolver entries for services
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     rc=0
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     for svc in "${!svc_ips[@]}"; do
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:       for ip in ${svc_ips[${svc}]}; do
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:         echo "${ip} ${svc} ${svc}.${CLUSTER_DOMAIN} # ${OPENSHIFT_MARKER}" >> "${TEMP_FILE}" || rc=$?
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:       done
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     done
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     if [[ $rc -ne 0 ]]; then
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:       sleep 60 & wait
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:       continue
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     fi
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     # TODO: Update /etc/hosts atomically to avoid any inconsistent behavior
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     # Replace /etc/hosts with our modified version if needed
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     cmp "${TEMP_FILE}" "${HOSTS_FILE}" || cp -f "${TEMP_FILE}" "${HOSTS_FILE}"
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:     # TEMP_FILE is not removed to avoid file create/delete and attributes copy churn
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   fi
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   sleep 60 & wait
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]:   unset svc_ips
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: done
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: ],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:SERVICES,Value:image-registry.openshift-image-registry.svc,ValueFrom:nil,},EnvVar{Name:NAMESERVER,Value:172.30.0.10,ValueFrom:nil,},EnvVar{Name:CLUSTER_DOMAIN,Value:cluster.local,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{5 -3} {<nil>} 5m DecimalSI},memory: {{22020096 0} {<nil>} 21Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:hosts-file,ReadOnly:false,MountPath:/etc/hosts,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:kube-api-access-g6vvr,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:*true,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:FallbackToLogsOnError,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod node-resolver-qngzk_openshift-dns(21b6dd11-c00e-4ec7-a192-703f8a824b80): ErrImagePull: pull QPS exceeded
Oct 25 18:54:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:52.819699    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-dns/node-resolver-qngzk" podUID="21b6dd11-c00e-4ec7-a192-703f8a824b80"
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.123955    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:53.124152    4293 secret.go:194] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:53.124237    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs podName:38bac755-4b7a-447a-bea0-6c1be8066cd8 nodeName:}" failed. No retries permitted until 2023-10-25 18:54:54.124218366 +0000 UTC m=+42.292948361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs") pod "network-metrics-daemon-ckj6x" (UID: "38bac755-4b7a-447a-bea0-6c1be8066cd8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.225092    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-n4n5q\" (UniqueName: \"kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q\") pod \"network-check-target-gg7qk\" (UID: \"2f285036-37bd-4419-a167-e41819a56c33\") " pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:53.225254    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:53.225276    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:53.225286    4293 projected.go:198] Error preparing data for projected volume kube-api-access-n4n5q for pod openshift-network-diagnostics/network-check-target-gg7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:53.225340    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q podName:2f285036-37bd-4419-a167-e41819a56c33 nodeName:}" failed. No retries permitted until 2023-10-25 18:54:54.225323955 +0000 UTC m=+42.394053950 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4n5q" (UniqueName: "kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q") pod "network-check-target-gg7qk" (UID: "2f285036-37bd-4419-a167-e41819a56c33") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:53.574766    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:53.574841    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.648285    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m" event={"ID":"ce1d733d-5d0c-42c3-93bf-254808356343","Type":"ContainerStarted","Data":"36edd484a40ed07995e9fa3d0c8ed9e25a9dc0d313e1160f30d41fe5d37c7db2"}
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.649744    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" event={"ID":"60df7285-d18c-4337-9e6b-27c667f60eee","Type":"ContainerStarted","Data":"f03e5eb4d48dd84ae23a7812bdb5e73a8c61ccad2f449e7c77bf5740d2ed9d87"}
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.650622    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" event={"ID":"aab63cd9-e2d8-4dff-8d13-22cb7a673703","Type":"ContainerStarted","Data":"a004f9f448933efced89461c741f879facaf81b713e0f8fc67549d3866d2d30a"}
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.651610    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qngzk" event={"ID":"21b6dd11-c00e-4ec7-a192-703f8a824b80","Type":"ContainerStarted","Data":"878a7752aba6f8cbfdc58b21858ba3e0abc8499187acbfbdf74da5460e268f9b"}
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:53.652979    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:6cfcf1895bb82eef1d4bcad458d207cb333e676f8f8155d7c7515d07b728ffed\\\"\"" pod="openshift-dns/node-resolver-qngzk" podUID="21b6dd11-c00e-4ec7-a192-703f8a824b80"
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.653752    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j4st5" event={"ID":"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2","Type":"ContainerStarted","Data":"c17087d9d7aeae01a96e0d0a54bd993a857d3290aa26e0fbd3f2effd501619c4"}
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.654603    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" event={"ID":"6a75dddb-d3b3-408f-9cab-46b51953ccc8","Type":"ContainerStarted","Data":"b27c7e05f2f68c88e5b205b6d5bce3305d4efbda9bba0f633ad04ea10eecbe1d"}
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.655377    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6mp47" event={"ID":"ce8d94be-2c75-42a5-a8e1-60275cefdcbd","Type":"ContainerStarted","Data":"98a3db0785877704f99c80581ed812e233d83013582159ac9ca292d61f8a3972"}
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.656207    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-29pm7" event={"ID":"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3","Type":"ContainerStarted","Data":"9de7d039c2a0cec9b42b46da0710b28b59ddb9e4da3f1b54f3f8d835f2ae9642"}
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.657110    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" event={"ID":"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b","Type":"ContainerStarted","Data":"b769947c845be1e63be96f4f0328209976ede68bab7248624161419c5c58de21"}
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.657908    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rfcb9" event={"ID":"9954521a-e814-4b76-b21a-3b08ad936c10","Type":"ContainerStarted","Data":"643df05148165c529d31e747ec90684bb8063416038b053118722e91c2cd5fae"}
Oct 25 18:54:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:53.658486    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qngxd" event={"ID":"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06","Type":"ContainerStarted","Data":"3ea09b6611caccacdb801c1626053a64420aa250d59e7cec4a626f91900ff3c2"}
Oct 25 18:54:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:54.130052    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:54:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:54.130222    4293 secret.go:194] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:54:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:54.130284    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs podName:38bac755-4b7a-447a-bea0-6c1be8066cd8 nodeName:}" failed. No retries permitted until 2023-10-25 18:54:56.13026765 +0000 UTC m=+44.298997645 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs") pod "network-metrics-daemon-ckj6x" (UID: "38bac755-4b7a-447a-bea0-6c1be8066cd8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:54:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:54.230853    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-n4n5q\" (UniqueName: \"kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q\") pod \"network-check-target-gg7qk\" (UID: \"2f285036-37bd-4419-a167-e41819a56c33\") " pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:54:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:54.230963    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 25 18:54:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:54.230979    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 25 18:54:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:54.230987    4293 projected.go:198] Error preparing data for projected volume kube-api-access-n4n5q for pod openshift-network-diagnostics/network-check-target-gg7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:54:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:54.231018    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q podName:2f285036-37bd-4419-a167-e41819a56c33 nodeName:}" failed. No retries permitted until 2023-10-25 18:54:56.231009693 +0000 UTC m=+44.399739688 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4n5q" (UniqueName: "kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q") pod "network-check-target-gg7qk" (UID: "2f285036-37bd-4419-a167-e41819a56c33") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:54:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:55.575211    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:54:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:55.575367    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:54:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:56.145153    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:54:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:56.145297    4293 secret.go:194] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:54:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:56.145360    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs podName:38bac755-4b7a-447a-bea0-6c1be8066cd8 nodeName:}" failed. No retries permitted until 2023-10-25 18:55:00.145342938 +0000 UTC m=+48.314072933 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs") pod "network-metrics-daemon-ckj6x" (UID: "38bac755-4b7a-447a-bea0-6c1be8066cd8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:54:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:54:56.246334    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-n4n5q\" (UniqueName: \"kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q\") pod \"network-check-target-gg7qk\" (UID: \"2f285036-37bd-4419-a167-e41819a56c33\") " pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:54:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:56.246523    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 25 18:54:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:56.246556    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 25 18:54:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:56.246569    4293 projected.go:198] Error preparing data for projected volume kube-api-access-n4n5q for pod openshift-network-diagnostics/network-check-target-gg7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:54:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:56.246633    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q podName:2f285036-37bd-4419-a167-e41819a56c33 nodeName:}" failed. No retries permitted until 2023-10-25 18:55:00.246614151 +0000 UTC m=+48.415344246 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4n5q" (UniqueName: "kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q") pod "network-check-target-gg7qk" (UID: "2f285036-37bd-4419-a167-e41819a56c33") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:54:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:57.574759    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:54:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:57.574883    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:54:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:59.574601    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:54:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:54:59.574739    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:00.189872    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:55:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:00.190004    4293 secret.go:194] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:55:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:00.190051    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs podName:38bac755-4b7a-447a-bea0-6c1be8066cd8 nodeName:}" failed. No retries permitted until 2023-10-25 18:55:08.190038846 +0000 UTC m=+56.358768841 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs") pod "network-metrics-daemon-ckj6x" (UID: "38bac755-4b7a-447a-bea0-6c1be8066cd8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:55:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:00.290886    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-n4n5q\" (UniqueName: \"kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q\") pod \"network-check-target-gg7qk\" (UID: \"2f285036-37bd-4419-a167-e41819a56c33\") " pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:55:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:00.291100    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 25 18:55:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:00.291148    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 25 18:55:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:00.291161    4293 projected.go:198] Error preparing data for projected volume kube-api-access-n4n5q for pod openshift-network-diagnostics/network-check-target-gg7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:55:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:00.291222    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q podName:2f285036-37bd-4419-a167-e41819a56c33 nodeName:}" failed. No retries permitted until 2023-10-25 18:55:08.291206897 +0000 UTC m=+56.459936892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4n5q" (UniqueName: "kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q") pod "network-check-target-gg7qk" (UID: "2f285036-37bd-4419-a167-e41819a56c33") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:55:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:01.575236    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:01.575384    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:03.575464    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:03.575599    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:05.574894    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:05.575027    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:07.574736    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:07.574767    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:08.249108    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:55:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:08.249267    4293 secret.go:194] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:55:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:08.249344    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs podName:38bac755-4b7a-447a-bea0-6c1be8066cd8 nodeName:}" failed. No retries permitted until 2023-10-25 18:55:24.249327118 +0000 UTC m=+72.418057113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs") pod "network-metrics-daemon-ckj6x" (UID: "38bac755-4b7a-447a-bea0-6c1be8066cd8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:55:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:08.349494    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-n4n5q\" (UniqueName: \"kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q\") pod \"network-check-target-gg7qk\" (UID: \"2f285036-37bd-4419-a167-e41819a56c33\") " pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:55:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:08.349611    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 25 18:55:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:08.349628    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 25 18:55:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:08.349636    4293 projected.go:198] Error preparing data for projected volume kube-api-access-n4n5q for pod openshift-network-diagnostics/network-check-target-gg7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:55:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:08.349682    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q podName:2f285036-37bd-4419-a167-e41819a56c33 nodeName:}" failed. No retries permitted until 2023-10-25 18:55:24.349668293 +0000 UTC m=+72.518398288 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4n5q" (UniqueName: "kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q") pod "network-check-target-gg7qk" (UID: "2f285036-37bd-4419-a167-e41819a56c33") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:55:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:09.575371    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:09.575529    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:11.574532    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:11.574645    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:12.406926    4293 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Running"
Oct 25 18:55:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:12.406986    4293 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Running"
Oct 25 18:55:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:13.575265    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:13.575298    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:15.575470    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:15.575532    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:17.575051    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:17.575187    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:19.575514    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:19.575619    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:21.575297    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:21.575476    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:23.575585    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:23.575683    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:24.256940    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:55:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:24.257079    4293 secret.go:194] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:55:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:24.257148    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs podName:38bac755-4b7a-447a-bea0-6c1be8066cd8 nodeName:}" failed. No retries permitted until 2023-10-25 18:55:56.257132193 +0000 UTC m=+104.425862188 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs") pod "network-metrics-daemon-ckj6x" (UID: "38bac755-4b7a-447a-bea0-6c1be8066cd8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 25 18:55:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:24.357549    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-n4n5q\" (UniqueName: \"kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q\") pod \"network-check-target-gg7qk\" (UID: \"2f285036-37bd-4419-a167-e41819a56c33\") " pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:55:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:24.357721    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 25 18:55:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:24.357752    4293 projected.go:292] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 25 18:55:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:24.357762    4293 projected.go:198] Error preparing data for projected volume kube-api-access-n4n5q for pod openshift-network-diagnostics/network-check-target-gg7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:55:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:24.357822    4293 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q podName:2f285036-37bd-4419-a167-e41819a56c33 nodeName:}" failed. No retries permitted until 2023-10-25 18:55:56.357807316 +0000 UTC m=+104.526537311 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n4n5q" (UniqueName: "kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q") pod "network-check-target-gg7qk" (UID: "2f285036-37bd-4419-a167-e41819a56c33") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:25.575361    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:25.575445    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:25.594720    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CreateContainerConfigError: \"services have not yet been read at least once, cannot construct envvars\"" pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" podUID="aab63cd9-e2d8-4dff-8d13-22cb7a673703"
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.822621    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rfcb9" event={"ID":"9954521a-e814-4b76-b21a-3b08ad936c10","Type":"ContainerStarted","Data":"97bdaa8671065a26a0dbf0cf492e614f30d7fea56c6f24752eb450bbe354501a"}
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.823744    4293 generic.go:334] "Generic (PLEG): container finished" podID="75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06" containerID="337d75ab8927e9ae0cf81aba5ccc1814a9a4bfa31a25802b835721bcccec4a92" exitCode=0
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.823829    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qngxd" event={"ID":"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06","Type":"ContainerDied","Data":"337d75ab8927e9ae0cf81aba5ccc1814a9a4bfa31a25802b835721bcccec4a92"}
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.825389    4293 generic.go:334] "Generic (PLEG): container finished" podID="6a75dddb-d3b3-408f-9cab-46b51953ccc8" containerID="8bb5c17007653caed3acd05295ca782ab245121d1c7c8ef4485cee6af31d6575" exitCode=0
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.825453    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" event={"ID":"6a75dddb-d3b3-408f-9cab-46b51953ccc8","Type":"ContainerDied","Data":"8bb5c17007653caed3acd05295ca782ab245121d1c7c8ef4485cee6af31d6575"}
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.827516    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6mp47" event={"ID":"ce8d94be-2c75-42a5-a8e1-60275cefdcbd","Type":"ContainerStarted","Data":"8fdf3529df6c8a31344b234a2e65d986b13d1c3fa8cada95dde3bca84eb5f599"}
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.829221    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j4st5" event={"ID":"93d972c7-c2ab-4f1e-8b56-d8d40be93cb2","Type":"ContainerStarted","Data":"90dd7d61f219f5c668e0a81359959092f316c41a305ddbaf270dc4b596dbb29c"}
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.830859    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m" event={"ID":"ce1d733d-5d0c-42c3-93bf-254808356343","Type":"ContainerStarted","Data":"a116a49b842e534d39399026911846a8724831fbadf5f0084925125161deb643"}
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.833559    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-29pm7" event={"ID":"39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3","Type":"ContainerStarted","Data":"72a3954f7a719092a389df5b84db0790b30a91285663095e53f45773b70e2eb0"}
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.835530    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" event={"ID":"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b","Type":"ContainerStarted","Data":"01e4007fcf7c467ef9a8e5fa57b4d25067bcf2539770a3488253310db55db203"}
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.836839    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" event={"ID":"aab63cd9-e2d8-4dff-8d13-22cb7a673703","Type":"ContainerStarted","Data":"55178b81e5e06b1070d6107b0448ea62d7bacf1d7d587d7b0094bc875bf82906"}
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.860968    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-multus/multus-j4st5" podStartSLOduration=7.519334002 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:54:52.790976568 +0000 UTC m=+40.959706563" lastFinishedPulling="2023-10-25 18:55:25.132553613 +0000 UTC m=+73.301283618" observedRunningTime="2023-10-25 18:55:25.860879626 +0000 UTC m=+74.029609631" watchObservedRunningTime="2023-10-25 18:55:25.860911057 +0000 UTC m=+74.029641052"
Oct 25 18:55:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:25.942175    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-manila-csi-driver/csi-nodeplugin-nfsplugin-j9n8m" podStartSLOduration=7.571998502 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:54:52.773630202 +0000 UTC m=+40.942360187" lastFinishedPulling="2023-10-25 18:55:25.143765814 +0000 UTC m=+73.312495809" observedRunningTime="2023-10-25 18:55:25.917534163 +0000 UTC m=+74.086264168" watchObservedRunningTime="2023-10-25 18:55:25.942134124 +0000 UTC m=+74.110864139"
Oct 25 18:55:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:26.841696    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" event={"ID":"aab63cd9-e2d8-4dff-8d13-22cb7a673703","Type":"ContainerStarted","Data":"d026498683eea5d70dec53cf0446acf6be5fb1e5e1930d978c9e30bec4881b41"}
Oct 25 18:55:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:26.843943    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" event={"ID":"6a75dddb-d3b3-408f-9cab-46b51953ccc8","Type":"ContainerStarted","Data":"937688d404317c1eebf3d28195f76897cdeb943e6780bf1ab3eb3425e4c361c1"}
Oct 25 18:55:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:26.843975    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" event={"ID":"6a75dddb-d3b3-408f-9cab-46b51953ccc8","Type":"ContainerStarted","Data":"07590a0f59f3834f9c17a35806c0c7fbec4b2f29765bc57266fad7c99d048b88"}
Oct 25 18:55:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:26.845041    4293 generic.go:334] "Generic (PLEG): container finished" podID="ce8d94be-2c75-42a5-a8e1-60275cefdcbd" containerID="8fdf3529df6c8a31344b234a2e65d986b13d1c3fa8cada95dde3bca84eb5f599" exitCode=0
Oct 25 18:55:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:26.845080    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6mp47" event={"ID":"ce8d94be-2c75-42a5-a8e1-60275cefdcbd","Type":"ContainerDied","Data":"8fdf3529df6c8a31344b234a2e65d986b13d1c3fa8cada95dde3bca84eb5f599"}
Oct 25 18:55:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:26.846752    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" event={"ID":"60df7285-d18c-4337-9e6b-27c667f60eee","Type":"ContainerStarted","Data":"908d100a7e7a069968b22750e083589b5f84657528f77b2fb17a97fecbf1412e"}
Oct 25 18:55:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:26.855278    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-machine-config-operator/machine-config-daemon-zmn8s" podStartSLOduration=10.229457189 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:54:52.804383638 +0000 UTC m=+40.973113633" lastFinishedPulling="2023-10-25 18:55:23.430177037 +0000 UTC m=+71.598907042" observedRunningTime="2023-10-25 18:55:26.85508049 +0000 UTC m=+75.023810495" watchObservedRunningTime="2023-10-25 18:55:26.855250598 +0000 UTC m=+75.023980593"
Oct 25 18:55:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:26.855495    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-29pm7" podStartSLOduration=8.534051749 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:54:52.806667982 +0000 UTC m=+40.975397977" lastFinishedPulling="2023-10-25 18:55:25.128097783 +0000 UTC m=+73.296827778" observedRunningTime="2023-10-25 18:55:25.957885489 +0000 UTC m=+74.126615484" watchObservedRunningTime="2023-10-25 18:55:26.85548155 +0000 UTC m=+75.024211545"
Oct 25 18:55:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:26.886406    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rfcb9" podStartSLOduration=10.253624485 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:54:52.797469594 +0000 UTC m=+40.966199579" lastFinishedPulling="2023-10-25 18:55:23.430223839 +0000 UTC m=+71.598953844" observedRunningTime="2023-10-25 18:55:26.868801135 +0000 UTC m=+75.037531130" watchObservedRunningTime="2023-10-25 18:55:26.88637875 +0000 UTC m=+75.055108735"
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.562139    4293 state_mem.go:80] "Updated desired CPUSet" podUID="93d972c7-c2ab-4f1e-8b56-d8d40be93cb2" containerName="kube-multus" cpuSet="0-7"
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:27.574739    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:27.574943    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.589301    4293 state_mem.go:80] "Updated desired CPUSet" podUID="39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3" containerName="tuned" cpuSet="0-7"
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.625705    4293 state_mem.go:80] "Updated desired CPUSet" podUID="9954521a-e814-4b76-b21a-3b08ad936c10" containerName="node-ca" cpuSet="0-7"
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.657342    4293 state_mem.go:80] "Updated desired CPUSet" podUID="aab63cd9-e2d8-4dff-8d13-22cb7a673703" containerName="machine-config-daemon" cpuSet="0-7"
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.687707    4293 state_mem.go:80] "Updated desired CPUSet" podUID="aab63cd9-e2d8-4dff-8d13-22cb7a673703" containerName="kube-rbac-proxy" cpuSet="0-7"
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.712556    4293 state_mem.go:80] "Updated desired CPUSet" podUID="ce1d733d-5d0c-42c3-93bf-254808356343" containerName="csi-driver" cpuSet="0-7"
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.852699    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qngzk" event={"ID":"21b6dd11-c00e-4ec7-a192-703f8a824b80","Type":"ContainerStarted","Data":"10b9ac3b4149944fac507371bfc7e32feba4d4b37dedfefd8a6db847d7194230"}
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.856415    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" event={"ID":"6a75dddb-d3b3-408f-9cab-46b51953ccc8","Type":"ContainerStarted","Data":"2dfb6e7231216aa7ab5aa11249482032a941d9ad295f434ba6837f8b0bc68ebf"}
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.856444    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" event={"ID":"6a75dddb-d3b3-408f-9cab-46b51953ccc8","Type":"ContainerStarted","Data":"37c411a4053d5dbf3b72696364c642444c5385b85235a0a086b93fd7591a724b"}
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.856460    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" event={"ID":"6a75dddb-d3b3-408f-9cab-46b51953ccc8","Type":"ContainerStarted","Data":"ee02858cbfd49b94e187c48ee841155f8e84b317b3e09995494b93f9db8dd85b"}
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.856472    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" event={"ID":"6a75dddb-d3b3-408f-9cab-46b51953ccc8","Type":"ContainerStarted","Data":"5ed1f634549545d1705b7e5379dcc68ca0d5cc1c68f8805d01b3b0dde20844f3"}
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.858126    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6mp47" event={"ID":"ce8d94be-2c75-42a5-a8e1-60275cefdcbd","Type":"ContainerStarted","Data":"edc25fbb099ca9d14a049eedbad05a28947b23844835ef627d5bf93a1ae2e5c8"}
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.858152    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6mp47" event={"ID":"ce8d94be-2c75-42a5-a8e1-60275cefdcbd","Type":"ContainerStarted","Data":"ac716f12fa73e0969e8d2adfb2c58181bd164e1612fa0dd192c6b7cef75fd161"}
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.864929    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-dns/node-resolver-qngzk" podStartSLOduration=7.9595785580000005 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:54:52.819573862 +0000 UTC m=+40.988303857" lastFinishedPulling="2023-10-25 18:55:26.724890275 +0000 UTC m=+74.893620270" observedRunningTime="2023-10-25 18:55:27.864414858 +0000 UTC m=+76.033144863" watchObservedRunningTime="2023-10-25 18:55:27.864894971 +0000 UTC m=+76.033624966"
Oct 25 18:55:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:27.882434    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6mp47" podStartSLOduration=9.543363398 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:54:52.801873319 +0000 UTC m=+40.970603314" lastFinishedPulling="2023-10-25 18:55:25.140900483 +0000 UTC m=+73.309630478" observedRunningTime="2023-10-25 18:55:27.880647946 +0000 UTC m=+76.049377931" watchObservedRunningTime="2023-10-25 18:55:27.882390562 +0000 UTC m=+76.051120567"
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.185377    4293 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/manila.csi.openstack.org-reg.sock"
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.210736    4293 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/cinder.csi.openstack.org-reg.sock"
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.544066    4293 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/manila.csi.openstack.org-reg.sock","Timestamp":"2023-10-25T18:55:29.185397169Z","Handler":null,"Name":""}
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.544342    4293 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/cinder.csi.openstack.org-reg.sock","Timestamp":"2023-10-25T18:55:29.210753367Z","Handler":null,"Name":""}
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.546144    4293 csi_plugin.go:99] kubernetes.io/csi: Trying to validate a new CSI Driver with name: manila.csi.openstack.org endpoint: /var/lib/kubelet/plugins/manila.csi.openstack.org/csi.sock versions: 1.0.0
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.546174    4293 csi_plugin.go:112] kubernetes.io/csi: Register new plugin with name: manila.csi.openstack.org at endpoint: /var/lib/kubelet/plugins/manila.csi.openstack.org/csi.sock
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.546239    4293 csi_plugin.go:99] kubernetes.io/csi: Trying to validate a new CSI Driver with name: cinder.csi.openstack.org endpoint: /var/lib/kubelet/plugins/cinder.csi.openstack.org/csi.sock versions: 1.0.0
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.546251    4293 csi_plugin.go:112] kubernetes.io/csi: Register new plugin with name: cinder.csi.openstack.org at endpoint: /var/lib/kubelet/plugins/cinder.csi.openstack.org/csi.sock
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:29.575252    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:29.575290    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.863209    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" event={"ID":"60df7285-d18c-4337-9e6b-27c667f60eee","Type":"ContainerStarted","Data":"f458c0b46aaaa5d4b0ddeab27d5c3291d976e42d19d7884cdf175c0a21720381"}
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.864340    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" event={"ID":"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b","Type":"ContainerStarted","Data":"d18a9313395f3b183c6e5e74051cc63a702308810e4be1ea7972815d3c267c7a"}
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.866921    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" event={"ID":"6a75dddb-d3b3-408f-9cab-46b51953ccc8","Type":"ContainerStarted","Data":"356844e8ce56b8dd0a0a991bb5e77242808bd0d0633ccfb6da7af9e2df753b15"}
Oct 25 18:55:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:29.881565    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-manila-csi-driver/openstack-manila-csi-nodeplugin-nclxm" podStartSLOduration=7.752595294 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:54:52.802205036 +0000 UTC m=+40.970935041" lastFinishedPulling="2023-10-25 18:55:28.931149221 +0000 UTC m=+77.099879216" observedRunningTime="2023-10-25 18:55:29.881508517 +0000 UTC m=+78.050238502" watchObservedRunningTime="2023-10-25 18:55:29.881539469 +0000 UTC m=+78.050269464"
Oct 25 18:55:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:31.574707    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:31.574840    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:32.747728    4293 state_mem.go:80] "Updated desired CPUSet" podUID="60df7285-d18c-4337-9e6b-27c667f60eee" containerName="csi-driver" cpuSet="0-7"
Oct 25 18:55:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:32.772335    4293 state_mem.go:80] "Updated desired CPUSet" podUID="60df7285-d18c-4337-9e6b-27c667f60eee" containerName="csi-node-driver-registrar" cpuSet="0-7"
Oct 25 18:55:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:32.793045    4293 state_mem.go:80] "Updated desired CPUSet" podUID="ce8d94be-2c75-42a5-a8e1-60275cefdcbd" containerName="node-exporter" cpuSet="0-7"
Oct 25 18:55:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:32.803640    4293 state_mem.go:80] "Updated desired CPUSet" podUID="ce8d94be-2c75-42a5-a8e1-60275cefdcbd" containerName="kube-rbac-proxy" cpuSet="0-7"
Oct 25 18:55:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:32.814605    4293 state_mem.go:80] "Updated desired CPUSet" podUID="21b6dd11-c00e-4ec7-a192-703f8a824b80" containerName="dns-node-resolver" cpuSet="0-7"
Oct 25 18:55:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:33.680392    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:33.680608    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:35.574806    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:35.574925    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:37.575444    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:37.575624    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:39.574908    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:39.575059    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:40.891156    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" event={"ID":"6a75dddb-d3b3-408f-9cab-46b51953ccc8","Type":"ContainerStarted","Data":"1c42638de6958fbb02c0b57e291b5556e7658a1d986140dcdfc68f57d64cab2f"}
Oct 25 18:55:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:40.891432    4293 kubelet.go:2503] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:55:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:40.892785    4293 generic.go:334] "Generic (PLEG): container finished" podID="75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06" containerID="712c3bf63e2eafe2b2013c3f739a8d91073bd369a40bf8403bfb319fcdd1db70" exitCode=0
Oct 25 18:55:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:40.892852    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qngxd" event={"ID":"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06","Type":"ContainerDied","Data":"712c3bf63e2eafe2b2013c3f739a8d91073bd369a40bf8403bfb319fcdd1db70"}
Oct 25 18:55:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:40.895057    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" event={"ID":"fdf0522a-ef9e-45e0-bee2-397d22cc2f3b","Type":"ContainerStarted","Data":"891528896b646f0a984d3909e1559fe955a55dd9400399936c2f7d50d799e7eb"}
Oct 25 18:55:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:40.916515    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f" podStartSLOduration=22.600608237 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:54:52.812871003 +0000 UTC m=+40.981600998" lastFinishedPulling="2023-10-25 18:55:25.128750796 +0000 UTC m=+73.297480801" observedRunningTime="2023-10-25 18:55:40.916173812 +0000 UTC m=+89.084903827" watchObservedRunningTime="2023-10-25 18:55:40.91648804 +0000 UTC m=+89.085218035"
Oct 25 18:55:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:41.574709    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:41.574940    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:41.901157    4293 kubelet.go:2503] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:55:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:41.901194    4293 kubelet.go:2503] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:55:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:42.865321    4293 state_mem.go:80] "Updated desired CPUSet" podUID="6a75dddb-d3b3-408f-9cab-46b51953ccc8" containerName="ovn-controller" cpuSet="0-7"
Oct 25 18:55:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:42.916869    4293 state_mem.go:80] "Updated desired CPUSet" podUID="6a75dddb-d3b3-408f-9cab-46b51953ccc8" containerName="ovn-acl-logging" cpuSet="0-7"
Oct 25 18:55:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:42.987520    4293 state_mem.go:80] "Updated desired CPUSet" podUID="6a75dddb-d3b3-408f-9cab-46b51953ccc8" containerName="kube-rbac-proxy-node" cpuSet="0-7"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.003609    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/openstack-cinder-csi-driver-node-p7lgj" podStartSLOduration=9.170622462 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:54:52.80189266 +0000 UTC m=+40.970622655" lastFinishedPulling="2023-10-25 18:55:40.634830359 +0000 UTC m=+88.803560354" observedRunningTime="2023-10-25 18:55:40.945931573 +0000 UTC m=+89.114661588" watchObservedRunningTime="2023-10-25 18:55:43.003560161 +0000 UTC m=+91.172290156"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.004532    4293 kubelet.go:2406] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ckj6x"]
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:43.004759    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.007771    4293 kubelet.go:2406] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gg7qk"]
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:43.008004    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.015431    4293 state_mem.go:80] "Updated desired CPUSet" podUID="6a75dddb-d3b3-408f-9cab-46b51953ccc8" containerName="kube-rbac-proxy-ovn-metrics" cpuSet="0-7"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.043458    4293 state_mem.go:80] "Updated desired CPUSet" podUID="6a75dddb-d3b3-408f-9cab-46b51953ccc8" containerName="northd" cpuSet="0-7"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.067002    4293 state_mem.go:80] "Updated desired CPUSet" podUID="6a75dddb-d3b3-408f-9cab-46b51953ccc8" containerName="nbdb" cpuSet="0-7"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.088051    4293 state_mem.go:80] "Updated desired CPUSet" podUID="6a75dddb-d3b3-408f-9cab-46b51953ccc8" containerName="sbdb" cpuSet="0-7"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.120263    4293 state_mem.go:80] "Updated desired CPUSet" podUID="6a75dddb-d3b3-408f-9cab-46b51953ccc8" containerName="ovnkube-controller" cpuSet="0-7"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.149562    4293 state_mem.go:80] "Updated desired CPUSet" podUID="fdf0522a-ef9e-45e0-bee2-397d22cc2f3b" containerName="csi-driver" cpuSet="0-7"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.172301    4293 state_mem.go:80] "Updated desired CPUSet" podUID="fdf0522a-ef9e-45e0-bee2-397d22cc2f3b" containerName="node-driver-registrar" cpuSet="0-7"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.228183    4293 state_mem.go:80] "Updated desired CPUSet" podUID="fdf0522a-ef9e-45e0-bee2-397d22cc2f3b" containerName="csi-liveness-probe" cpuSet="0-7"
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.907099    4293 generic.go:334] "Generic (PLEG): container finished" podID="75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06" containerID="97f9b827361988cc0915981748018e19609702fbdaf011a6982298076027bc80" exitCode=0
Oct 25 18:55:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:43.907137    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qngxd" event={"ID":"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06","Type":"ContainerDied","Data":"97f9b827361988cc0915981748018e19609702fbdaf011a6982298076027bc80"}
Oct 25 18:55:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:44.576386    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg7qk" podUID="2f285036-37bd-4419-a167-e41819a56c33"
Oct 25 18:55:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: E1025 18:55:44.576704    4293 pod_workers.go:1300] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckj6x" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.255300    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeReady"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.255636    4293 kubelet_node_status.go:520] "Fast updating node status as it just became ready"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319480    4293 kubelet.go:2399] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2ncqb"]
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319518    4293 topology_manager.go:215] "Topology Admit Handler" podUID="dbed798e-c84c-4287-b7e4-aefe4a08c3cb" podNamespace="openshift-dns" podName="dns-default-2ncqb"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319540    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-dns/dns-default-2ncqb" containerName="dns"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319599    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-dns/dns-default-2ncqb" containerName="dns"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319629    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-dns/dns-default-2ncqb" containerName="dns"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319636    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319641    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319646    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319658    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319664    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-dns/dns-default-2ncqb" containerName="dns"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319669    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-dns/dns-default-2ncqb" containerName="dns"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319727    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-dns/dns-default-2ncqb" containerName="kube-rbac-proxy"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319753    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-dns/dns-default-2ncqb" containerName="kube-rbac-proxy"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319780    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-dns/dns-default-2ncqb" containerName="kube-rbac-proxy"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319785    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319791    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319795    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319800    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319806    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-dns/dns-default-2ncqb" containerName="kube-rbac-proxy"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.319812    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-dns/dns-default-2ncqb" containerName="kube-rbac-proxy"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.326313    4293 kubelet.go:2399] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-54pq9"]
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.326340    4293 topology_manager.go:215] "Topology Admit Handler" podUID="de1fef77-9a99-40d0-a88a-9a1d851e832f" podNamespace="openshift-ingress-canary" podName="ingress-canary-54pq9"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.326360    4293 scope_container.go:75] "TopologyHints" hints={} pod="openshift-ingress-canary/ingress-canary-54pq9" containerName="serve-healthcheck-canary"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.326396    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ingress-canary/ingress-canary-54pq9" containerName="serve-healthcheck-canary"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.326423    4293 scope_container.go:75] "TopologyHints" hints=null pod="openshift-ingress-canary/ingress-canary-54pq9" containerName="serve-healthcheck-canary"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.326429    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.326434    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.326438    4293 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.326446    4293 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.326451    4293 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ingress-canary/ingress-canary-54pq9" containerName="serve-healthcheck-canary"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.326457    4293 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-ingress-canary/ingress-canary-54pq9" containerName="serve-healthcheck-canary"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.333769    4293 kubelet.go:2406] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2ncqb"]
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.342874    4293 kubelet.go:2406] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-54pq9"]
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.512150    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tbfh\" (UniqueName: \"kubernetes.io/projected/de1fef77-9a99-40d0-a88a-9a1d851e832f-kube-api-access-5tbfh\") pod \"ingress-canary-54pq9\" (UID: \"de1fef77-9a99-40d0-a88a-9a1d851e832f\") " pod="openshift-ingress-canary/ingress-canary-54pq9"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.512196    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-config-volume\") pod \"dns-default-2ncqb\" (UID: \"dbed798e-c84c-4287-b7e4-aefe4a08c3cb\") " pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.512217    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2466z\" (UniqueName: \"kubernetes.io/projected/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-kube-api-access-2466z\") pod \"dns-default-2ncqb\" (UID: \"dbed798e-c84c-4287-b7e4-aefe4a08c3cb\") " pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.512385    4293 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-metrics-tls\") pod \"dns-default-2ncqb\" (UID: \"dbed798e-c84c-4287-b7e4-aefe4a08c3cb\") " pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.613693    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-5tbfh\" (UniqueName: \"kubernetes.io/projected/de1fef77-9a99-40d0-a88a-9a1d851e832f-kube-api-access-5tbfh\") pod \"ingress-canary-54pq9\" (UID: \"de1fef77-9a99-40d0-a88a-9a1d851e832f\") " pod="openshift-ingress-canary/ingress-canary-54pq9"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.613772    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-config-volume\") pod \"dns-default-2ncqb\" (UID: \"dbed798e-c84c-4287-b7e4-aefe4a08c3cb\") " pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.613796    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-2466z\" (UniqueName: \"kubernetes.io/projected/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-kube-api-access-2466z\") pod \"dns-default-2ncqb\" (UID: \"dbed798e-c84c-4287-b7e4-aefe4a08c3cb\") " pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.614157    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-metrics-tls\") pod \"dns-default-2ncqb\" (UID: \"dbed798e-c84c-4287-b7e4-aefe4a08c3cb\") " pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.614512    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-config-volume\") pod \"dns-default-2ncqb\" (UID: \"dbed798e-c84c-4287-b7e4-aefe4a08c3cb\") " pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.618709    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-metrics-tls\") pod \"dns-default-2ncqb\" (UID: \"dbed798e-c84c-4287-b7e4-aefe4a08c3cb\") " pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.640130    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tbfh\" (UniqueName: \"kubernetes.io/projected/de1fef77-9a99-40d0-a88a-9a1d851e832f-kube-api-access-5tbfh\") pod \"ingress-canary-54pq9\" (UID: \"de1fef77-9a99-40d0-a88a-9a1d851e832f\") " pod="openshift-ingress-canary/ingress-canary-54pq9"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.648628    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-54pq9"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.652089    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-2466z\" (UniqueName: \"kubernetes.io/projected/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-kube-api-access-2466z\") pod \"dns-default-2ncqb\" (UID: \"dbed798e-c84c-4287-b7e4-aefe4a08c3cb\") " pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.799010    4293 kubelet.go:2406] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-54pq9"]
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:55:46.942338    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde1fef77_9a99_40d0_a88a_9a1d851e832f.slice/crio-01e0c96ddfe168a54b96a2a706051e2ea38efb005e8588ecd109846e5e58611e WatchSource:0}: Error finding container 01e0c96ddfe168a54b96a2a706051e2ea38efb005e8588ecd109846e5e58611e: Status 404 returned error can't find the container with id 01e0c96ddfe168a54b96a2a706051e2ea38efb005e8588ecd109846e5e58611e
Oct 25 18:55:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:46.946246    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:47.045641    4293 kubelet.go:2406] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2ncqb"]
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:55:47.059851    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbed798e_c84c_4287_b7e4_aefe4a08c3cb.slice/crio-45fc9c58eb4fc0b494943a5e97c04a050f459e4df156675f49a4400f90e1017f WatchSource:0}: Error finding container 45fc9c58eb4fc0b494943a5e97c04a050f459e4df156675f49a4400f90e1017f: Status 404 returned error can't find the container with id 45fc9c58eb4fc0b494943a5e97c04a050f459e4df156675f49a4400f90e1017f
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:47.591383    4293 kubelet.go:2399] "SyncLoop ADD" source="api" pods=["openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"]
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:47.919628    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ncqb" event={"ID":"dbed798e-c84c-4287-b7e4-aefe4a08c3cb","Type":"ContainerStarted","Data":"fa3de44c284172eba747191e2fbaf72eefbb2948c6c128eeee366afbeaacf7b2"}
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:47.919665    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ncqb" event={"ID":"dbed798e-c84c-4287-b7e4-aefe4a08c3cb","Type":"ContainerStarted","Data":"0c518337c4fca4ba96e92792b62d56a921ad82a7f54aff8e40e19e9f077d179d"}
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:47.919683    4293 kubelet.go:2503] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:47.919696    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ncqb" event={"ID":"dbed798e-c84c-4287-b7e4-aefe4a08c3cb","Type":"ContainerStarted","Data":"45fc9c58eb4fc0b494943a5e97c04a050f459e4df156675f49a4400f90e1017f"}
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:47.921806    4293 generic.go:334] "Generic (PLEG): container finished" podID="75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06" containerID="5997bb4f5f2456ab12b3e3c5c3a203a2f635a844559f740f5c5c77dbdb5140ef" exitCode=0
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:47.921872    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qngxd" event={"ID":"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06","Type":"ContainerDied","Data":"5997bb4f5f2456ab12b3e3c5c3a203a2f635a844559f740f5c5c77dbdb5140ef"}
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:47.923073    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-54pq9" event={"ID":"de1fef77-9a99-40d0-a88a-9a1d851e832f","Type":"ContainerStarted","Data":"01e0c96ddfe168a54b96a2a706051e2ea38efb005e8588ecd109846e5e58611e"}
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:47.933926    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-dns/dns-default-2ncqb" podStartSLOduration=1.933900316 podCreationTimestamp="2023-10-25 18:55:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2023-10-25 18:55:47.933540297 +0000 UTC m=+96.102270292" watchObservedRunningTime="2023-10-25 18:55:47.933900316 +0000 UTC m=+96.102630311"
Oct 25 18:55:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:47.946677    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podStartSLOduration=0.946650951 podCreationTimestamp="2023-10-25 18:55:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2023-10-25 18:55:47.946308132 +0000 UTC m=+96.115038127" watchObservedRunningTime="2023-10-25 18:55:47.946650951 +0000 UTC m=+96.115380946"
Oct 25 18:55:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:48.254136    4293 state_mem.go:80] "Updated desired CPUSet" podUID="dbed798e-c84c-4287-b7e4-aefe4a08c3cb" containerName="dns" cpuSet="0-7"
Oct 25 18:55:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:48.266232    4293 state_mem.go:80] "Updated desired CPUSet" podUID="dbed798e-c84c-4287-b7e4-aefe4a08c3cb" containerName="kube-rbac-proxy" cpuSet="0-7"
Oct 25 18:55:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:50.930661    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-54pq9" event={"ID":"de1fef77-9a99-40d0-a88a-9a1d851e832f","Type":"ContainerStarted","Data":"7649f55e35eb1e40b8ef1bfbad4f9e3af785146761c77e13a5d96b6187f7c14a"}
Oct 25 18:55:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:52.821497    4293 kubelet.go:2503] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wc25f"
Oct 25 18:55:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:52.846024    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-54pq9" podStartSLOduration=3.301310264 podCreationTimestamp="2023-10-25 18:55:46 +0000 UTC" firstStartedPulling="2023-10-25 18:55:46.944636759 +0000 UTC m=+95.113366754" lastFinishedPulling="2023-10-25 18:55:50.489304146 +0000 UTC m=+98.658034151" observedRunningTime="2023-10-25 18:55:50.944196692 +0000 UTC m=+99.112926687" watchObservedRunningTime="2023-10-25 18:55:52.845977661 +0000 UTC m=+101.014707656"
Oct 25 18:55:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:53.292414    4293 state_mem.go:80] "Updated desired CPUSet" podUID="de1fef77-9a99-40d0-a88a-9a1d851e832f" containerName="serve-healthcheck-canary" cpuSet="0-7"
Oct 25 18:55:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:54.941463    4293 generic.go:334] "Generic (PLEG): container finished" podID="75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06" containerID="e4fa551b1ca5b5fa1c339de1ee4870fb95796a6803d4726c188af9422fb877f8" exitCode=0
Oct 25 18:55:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:54.941504    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qngxd" event={"ID":"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06","Type":"ContainerDied","Data":"e4fa551b1ca5b5fa1c339de1ee4870fb95796a6803d4726c188af9422fb877f8"}
Oct 25 18:55:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:55.946542    4293 generic.go:334] "Generic (PLEG): container finished" podID="75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06" containerID="01879b09ca040de594aae8d8600d35fef632bb1c6f336c9136219353de0cb520" exitCode=0
Oct 25 18:55:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:55.946606    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qngxd" event={"ID":"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06","Type":"ContainerDied","Data":"01879b09ca040de594aae8d8600d35fef632bb1c6f336c9136219353de0cb520"}
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.285457    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.288898    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs\") pod \"network-metrics-daemon-ckj6x\" (UID: \"38bac755-4b7a-447a-bea0-6c1be8066cd8\") " pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.386112    4293 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kube-api-access-n4n5q\" (UniqueName: \"kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q\") pod \"network-check-target-gg7qk\" (UID: \"2f285036-37bd-4419-a167-e41819a56c33\") " pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.388980    4293 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4n5q\" (UniqueName: \"kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q\") pod \"network-check-target-gg7qk\" (UID: \"2f285036-37bd-4419-a167-e41819a56c33\") " pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.500159    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.502183    4293 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckj6x"
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.629445    4293 kubelet.go:2406] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ckj6x"]
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.638944    4293 kubelet.go:2406] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gg7qk"]
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:55:56.645771    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bac755_4b7a_447a_bea0_6c1be8066cd8.slice/crio-09814fb7053b5874dad08247b7694ad5cec3eb08717c074b316f2ecaac8ebae3 WatchSource:0}: Error finding container 09814fb7053b5874dad08247b7694ad5cec3eb08717c074b316f2ecaac8ebae3: Status 404 returned error can't find the container with id 09814fb7053b5874dad08247b7694ad5cec3eb08717c074b316f2ecaac8ebae3
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: W1025 18:55:56.645985    4293 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f285036_37bd_4419_a167_e41819a56c33.slice/crio-9ef973628c0d01f3e123c4cab4825dd4a5474af47b22f995fbd3b49eb582cf51 WatchSource:0}: Error finding container 9ef973628c0d01f3e123c4cab4825dd4a5474af47b22f995fbd3b49eb582cf51: Status 404 returned error can't find the container with id 9ef973628c0d01f3e123c4cab4825dd4a5474af47b22f995fbd3b49eb582cf51
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.949617    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gg7qk" event={"ID":"2f285036-37bd-4419-a167-e41819a56c33","Type":"ContainerStarted","Data":"9ef973628c0d01f3e123c4cab4825dd4a5474af47b22f995fbd3b49eb582cf51"}
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.950629    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ckj6x" event={"ID":"38bac755-4b7a-447a-bea0-6c1be8066cd8","Type":"ContainerStarted","Data":"09814fb7053b5874dad08247b7694ad5cec3eb08717c074b316f2ecaac8ebae3"}
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.953287    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qngxd" event={"ID":"75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06","Type":"ContainerStarted","Data":"a241ca6b94130c23cd35c76df635c41c784cdd545dbff804c918c5fe8115fb94"}
Oct 25 18:55:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:56.979434    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qngxd" podStartSLOduration=9.812755533 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:54:52.791039479 +0000 UTC m=+40.959769474" lastFinishedPulling="2023-10-25 18:55:53.957672441 +0000 UTC m=+102.126402426" observedRunningTime="2023-10-25 18:55:56.97423399 +0000 UTC m=+105.142963985" watchObservedRunningTime="2023-10-25 18:55:56.979388485 +0000 UTC m=+105.148118480"
Oct 25 18:55:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:58.316551    4293 state_mem.go:80] "Updated desired CPUSet" podUID="75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06" containerName="kube-multus-additional-cni-plugins" cpuSet="0-7"
Oct 25 18:55:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:58.948544    4293 kubelet.go:2503] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2ncqb"
Oct 25 18:55:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:59.961573    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ckj6x" event={"ID":"38bac755-4b7a-447a-bea0-6c1be8066cd8","Type":"ContainerStarted","Data":"7640c606ef7c19ef6939b596e8b70868ec780831f64f5199f7a8a554082afae5"}
Oct 25 18:55:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:59.961874    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ckj6x" event={"ID":"38bac755-4b7a-447a-bea0-6c1be8066cd8","Type":"ContainerStarted","Data":"eebde62c49d18760bf4e64a95bd3bf16ae4e7057104c6f7d0d330af75ee7d28c"}
Oct 25 18:55:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:55:59.978609    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ckj6x" podStartSLOduration=71.230231878 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:55:56.649311545 +0000 UTC m=+104.818041540" lastFinishedPulling="2023-10-25 18:55:59.397645166 +0000 UTC m=+107.566375171" observedRunningTime="2023-10-25 18:55:59.978147738 +0000 UTC m=+108.146877763" watchObservedRunningTime="2023-10-25 18:55:59.978565509 +0000 UTC m=+108.147295514"
Oct 25 18:56:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:00.585259    4293 kubelet.go:2399] "SyncLoop ADD" source="api" pods=["openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"]
Oct 25 18:56:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:02.590488    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podStartSLOduration=2.5904368829999997 podCreationTimestamp="2023-10-25 18:56:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2023-10-25 18:56:02.588495932 +0000 UTC m=+110.757225927" watchObservedRunningTime="2023-10-25 18:56:02.590436883 +0000 UTC m=+110.759166888"
Oct 25 18:56:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:03.335229    4293 state_mem.go:80] "Updated desired CPUSet" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8" containerName="network-metrics-daemon" cpuSet="0-7"
Oct 25 18:56:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:03.357239    4293 state_mem.go:80] "Updated desired CPUSet" podUID="38bac755-4b7a-447a-bea0-6c1be8066cd8" containerName="kube-rbac-proxy" cpuSet="0-7"
Oct 25 18:56:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:03.969852    4293 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gg7qk" event={"ID":"2f285036-37bd-4419-a167-e41819a56c33","Type":"ContainerStarted","Data":"b8d4ab90a438eb715158a1a54e758ae755c92bf38ff27a67d5ed5ce52896b28a"}
Oct 25 18:56:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:03.970657    4293 kubelet.go:2503] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-network-diagnostics/network-check-target-gg7qk"
Oct 25 18:56:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:03.984169    4293 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gg7qk" podStartSLOduration=71.488859762 podCreationTimestamp="2023-10-25 18:54:46 +0000 UTC" firstStartedPulling="2023-10-25 18:55:56.651095041 +0000 UTC m=+104.819825026" lastFinishedPulling="2023-10-25 18:56:03.146379119 +0000 UTC m=+111.315109114" observedRunningTime="2023-10-25 18:56:03.981599693 +0000 UTC m=+112.150329688" watchObservedRunningTime="2023-10-25 18:56:03.98414385 +0000 UTC m=+112.152873845"
Oct 25 18:56:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:08.380533    4293 state_mem.go:80] "Updated desired CPUSet" podUID="2f285036-37bd-4419-a167-e41819a56c33" containerName="network-check-target-container" cpuSet="0-7"
Oct 25 18:56:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:12.407854    4293 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Running"
Oct 25 18:56:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:12.408331    4293 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Running"
Oct 25 18:56:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:28.941394    4293 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeNotSchedulable"
Oct 25 18:56:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:31.564538    4293 dynamic_cafile_content.go:211] "Failed to remove file watch, it may have been deleted" file="/etc/kubernetes/kubelet-ca.crt" err="can't remove non-existent watcher: /etc/kubernetes/kubelet-ca.crt"
Oct 25 18:56:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:31.992315    4293 plugin_watcher.go:206] "Removing socket path from desired state cache" path="/var/lib/kubelet/plugins_registry/cinder.csi.openstack.org-reg.sock"
Oct 25 18:56:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:31.992438    4293 patch_prober.go:28] interesting pod/dns-default-2ncqb container/dns namespace/openshift-dns: Readiness probe status=failure output="Get \"http://10.131.0.5:8181/ready\": dial tcp 10.131.0.5:8181: connect: connection refused" start-of-body=
Oct 25 18:56:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:31.992510    4293 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-dns/dns-default-2ncqb" podUID="dbed798e-c84c-4287-b7e4-aefe4a08c3cb" containerName="dns" probeResult="failure" output="Get \"http://10.131.0.5:8181/ready\": dial tcp 10.131.0.5:8181: connect: connection refused"
Oct 25 18:56:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4293]: I1025 18:56:32.017526    4293 plugin_watcher.go:206] "Removing socket path from desired state cache" path="/var/lib/kubelet/plugins_registry/manila.csi.openstack.org-reg.sock"
Oct 25 18:56:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn systemd[1]: Stopping Kubernetes Kubelet...
Oct 25 18:56:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn systemd[1]: kubelet.service: Deactivated successfully.
Oct 25 18:56:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn systemd[1]: Stopped Kubernetes Kubelet.
Oct 25 18:56:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn systemd[1]: kubelet.service: Consumed 6.353s CPU time.
-- Boot f703f2ce3884411983f5a24ce1d0aa20 --
Oct 25 18:58:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn systemd[1]: Starting Kubernetes Kubelet...
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: Flag --address has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: Flag --cloud-provider has been deprecated, will be removed in 1.25 or later, in favor of removing cloud provider code from Kubelet.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: Flag --pod-infra-container-image has been deprecated, will be removed in a future release. Image garbage collector will get sandbox image information from CRI.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.273269    4021 server.go:203] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291162    4021 feature_gate.go:227] unrecognized feature gate: AdminNetworkPolicy
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291173    4021 feature_gate.go:227] unrecognized feature gate: AutomatedEtcdBackup
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291180    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProvider
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291185    4021 feature_gate.go:227] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291191    4021 feature_gate.go:227] unrecognized feature gate: DNSNameResolver
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291196    4021 feature_gate.go:227] unrecognized feature gate: RouteExternalCertificate
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291202    4021 feature_gate.go:227] unrecognized feature gate: VSphereStaticIPs
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291208    4021 feature_gate.go:227] unrecognized feature gate: GatewayAPI
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291213    4021 feature_gate.go:227] unrecognized feature gate: BuildCSIVolumes
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291219    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderAzure
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291224    4021 feature_gate.go:227] unrecognized feature gate: AzureWorkloadIdentity
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291230    4021 feature_gate.go:227] unrecognized feature gate: PrivateHostedZoneAWS
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291235    4021 feature_gate.go:227] unrecognized feature gate: InsightsConfigAPI
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291241    4021 feature_gate.go:227] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291247    4021 feature_gate.go:227] unrecognized feature gate: AlibabaPlatform
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291252    4021 feature_gate.go:227] unrecognized feature gate: CSIDriverSharedResource
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291258    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderExternal
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291263    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderGCP
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291269    4021 feature_gate.go:227] unrecognized feature gate: GCPLabelsTags
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291274    4021 feature_gate.go:242] Setting GA feature gate RetroactiveDefaultStorageClass=false. It will be removed in a future release.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291280    4021 feature_gate.go:227] unrecognized feature gate: MachineAPIProviderOpenStack
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.291286    4021 feature_gate.go:227] unrecognized feature gate: SigstoreImageVerification
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291357    4021 flags.go:64] FLAG: --address="10.0.3.42"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291367    4021 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291378    4021 flags.go:64] FLAG: --anonymous-auth="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291387    4021 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291393    4021 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291397    4021 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291403    4021 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291409    4021 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291413    4021 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291417    4021 flags.go:64] FLAG: --azure-container-registry-config=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291421    4021 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291426    4021 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291431    4021 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291436    4021 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291440    4021 flags.go:64] FLAG: --cgroup-root=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291444    4021 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291449    4021 flags.go:64] FLAG: --client-ca-file=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291453    4021 flags.go:64] FLAG: --cloud-config=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291457    4021 flags.go:64] FLAG: --cloud-provider="external"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291463    4021 flags.go:64] FLAG: --cluster-dns="[]"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291468    4021 flags.go:64] FLAG: --cluster-domain=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291472    4021 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291477    4021 flags.go:64] FLAG: --config-dir=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291481    4021 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291486    4021 flags.go:64] FLAG: --container-log-max-files="5"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291492    4021 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291496    4021 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291501    4021 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291505    4021 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291510    4021 flags.go:64] FLAG: --contention-profiling="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291515    4021 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291519    4021 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291524    4021 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291528    4021 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291534    4021 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291538    4021 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291542    4021 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291547    4021 flags.go:64] FLAG: --enable-load-reader="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291551    4021 flags.go:64] FLAG: --enable-server="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291555    4021 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291561    4021 flags.go:64] FLAG: --event-burst="100"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291565    4021 flags.go:64] FLAG: --event-qps="50"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291570    4021 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291574    4021 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291578    4021 flags.go:64] FLAG: --eviction-hard=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291584    4021 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291588    4021 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291593    4021 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291597    4021 flags.go:64] FLAG: --eviction-soft=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291602    4021 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291606    4021 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291610    4021 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291614    4021 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291618    4021 flags.go:64] FLAG: --fail-swap-on="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291623    4021 flags.go:64] FLAG: --feature-gates=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291628    4021 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291632    4021 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291637    4021 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291643    4021 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291648    4021 flags.go:64] FLAG: --healthz-port="10248"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291653    4021 flags.go:64] FLAG: --help="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291657    4021 flags.go:64] FLAG: --hostname-override="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291662    4021 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291666    4021 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291670    4021 flags.go:64] FLAG: --image-credential-provider-bin-dir=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291674    4021 flags.go:64] FLAG: --image-credential-provider-config=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291680    4021 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291684    4021 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291688    4021 flags.go:64] FLAG: --image-service-endpoint=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291693    4021 flags.go:64] FLAG: --iptables-drop-bit="15"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291697    4021 flags.go:64] FLAG: --iptables-masquerade-bit="14"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291701    4021 flags.go:64] FLAG: --keep-terminated-pod-volumes="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291705    4021 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291709    4021 flags.go:64] FLAG: --kube-api-burst="100"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291713    4021 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291718    4021 flags.go:64] FLAG: --kube-api-qps="50"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291722    4021 flags.go:64] FLAG: --kube-reserved=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291726    4021 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291730    4021 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291735    4021 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291739    4021 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291743    4021 flags.go:64] FLAG: --lock-file=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291748    4021 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291752    4021 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291756    4021 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291763    4021 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291767    4021 flags.go:64] FLAG: --logging-format="text"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291772    4021 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291777    4021 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291781    4021 flags.go:64] FLAG: --manifest-url=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291786    4021 flags.go:64] FLAG: --manifest-url-header=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291792    4021 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291796    4021 flags.go:64] FLAG: --max-open-files="1000000"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291802    4021 flags.go:64] FLAG: --max-pods="110"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291806    4021 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291811    4021 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291815    4021 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291819    4021 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291823    4021 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291828    4021 flags.go:64] FLAG: --node-ip="10.0.3.42"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291832    4021 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291842    4021 flags.go:64] FLAG: --node-status-max-images="50"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291848    4021 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291852    4021 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291856    4021 flags.go:64] FLAG: --pod-cidr=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291860    4021 flags.go:64] FLAG: --pod-infra-container-image="registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:aa40faef7732804a99271483a83076a9573c8ecc6e029f7cdd2b4faa7bede925"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291869    4021 flags.go:64] FLAG: --pod-manifest-path=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291873    4021 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291878    4021 flags.go:64] FLAG: --pods-per-core="0"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291898    4021 flags.go:64] FLAG: --port="10250"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291903    4021 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291909    4021 flags.go:64] FLAG: --provider-id=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291913    4021 flags.go:64] FLAG: --qos-reserved=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291918    4021 flags.go:64] FLAG: --read-only-port="10255"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291922    4021 flags.go:64] FLAG: --register-node="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291926    4021 flags.go:64] FLAG: --register-schedulable="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291930    4021 flags.go:64] FLAG: --register-with-taints=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291936    4021 flags.go:64] FLAG: --registry-burst="10"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291940    4021 flags.go:64] FLAG: --registry-qps="5"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291944    4021 flags.go:64] FLAG: --reserved-cpus=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291948    4021 flags.go:64] FLAG: --reserved-memory=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291953    4021 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291958    4021 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291962    4021 flags.go:64] FLAG: --rotate-certificates="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291967    4021 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291971    4021 flags.go:64] FLAG: --runonce="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291975    4021 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291980    4021 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291984    4021 flags.go:64] FLAG: --seccomp-default="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291988    4021 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291992    4021 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.291997    4021 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292001    4021 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292005    4021 flags.go:64] FLAG: --storage-driver-password="root"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292009    4021 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292014    4021 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292019    4021 flags.go:64] FLAG: --storage-driver-user="root"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292023    4021 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292029    4021 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292033    4021 flags.go:64] FLAG: --system-cgroups=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292037    4021 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=500Mi"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292045    4021 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292049    4021 flags.go:64] FLAG: --tls-cert-file=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292054    4021 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292060    4021 flags.go:64] FLAG: --tls-min-version=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292064    4021 flags.go:64] FLAG: --tls-private-key-file=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292068    4021 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292072    4021 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292077    4021 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292081    4021 flags.go:64] FLAG: --v="2"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292087    4021 flags.go:64] FLAG: --version="false"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292092    4021 flags.go:64] FLAG: --vmodule=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292097    4021 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292102    4021 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292167    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderAzure
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292175    4021 feature_gate.go:227] unrecognized feature gate: AzureWorkloadIdentity
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292181    4021 feature_gate.go:227] unrecognized feature gate: PrivateHostedZoneAWS
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292187    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderExternal
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292192    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderGCP
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292198    4021 feature_gate.go:227] unrecognized feature gate: GCPLabelsTags
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292204    4021 feature_gate.go:227] unrecognized feature gate: InsightsConfigAPI
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292209    4021 feature_gate.go:227] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292215    4021 feature_gate.go:227] unrecognized feature gate: AlibabaPlatform
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292221    4021 feature_gate.go:227] unrecognized feature gate: CSIDriverSharedResource
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292227    4021 feature_gate.go:242] Setting GA feature gate RetroactiveDefaultStorageClass=false. It will be removed in a future release.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292233    4021 feature_gate.go:227] unrecognized feature gate: SigstoreImageVerification
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292239    4021 feature_gate.go:227] unrecognized feature gate: MachineAPIProviderOpenStack
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292245    4021 feature_gate.go:227] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292251    4021 feature_gate.go:227] unrecognized feature gate: AdminNetworkPolicy
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292256    4021 feature_gate.go:227] unrecognized feature gate: AutomatedEtcdBackup
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292261    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProvider
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292267    4021 feature_gate.go:227] unrecognized feature gate: DNSNameResolver
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292272    4021 feature_gate.go:227] unrecognized feature gate: RouteExternalCertificate
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292278    4021 feature_gate.go:227] unrecognized feature gate: VSphereStaticIPs
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292285    4021 feature_gate.go:227] unrecognized feature gate: GatewayAPI
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.292290    4021 feature_gate.go:227] unrecognized feature gate: BuildCSIVolumes
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.292296    4021 feature_gate.go:250] feature gates: &{map[CloudDualStackNodeIPs:true DynamicResourceAllocation:false EventedPLEG:false MaxUnavailableStatefulSet:false NodeSwap:false RetroactiveDefaultStorageClass:false ValidatingAdmissionPolicy:false]}
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.310455    4021 server.go:467] "Kubelet version" kubeletVersion="v1.28.2+d981348"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.310472    4021 server.go:469] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310519    4021 feature_gate.go:227] unrecognized feature gate: BuildCSIVolumes
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310526    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderAzure
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310533    4021 feature_gate.go:227] unrecognized feature gate: AzureWorkloadIdentity
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310538    4021 feature_gate.go:227] unrecognized feature gate: PrivateHostedZoneAWS
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310544    4021 feature_gate.go:227] unrecognized feature gate: InsightsConfigAPI
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310550    4021 feature_gate.go:227] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310555    4021 feature_gate.go:227] unrecognized feature gate: AlibabaPlatform
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310561    4021 feature_gate.go:227] unrecognized feature gate: CSIDriverSharedResource
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310567    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderExternal
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310573    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderGCP
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310578    4021 feature_gate.go:227] unrecognized feature gate: GCPLabelsTags
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310584    4021 feature_gate.go:242] Setting GA feature gate RetroactiveDefaultStorageClass=false. It will be removed in a future release.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310596    4021 feature_gate.go:227] unrecognized feature gate: MachineAPIProviderOpenStack
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310601    4021 feature_gate.go:227] unrecognized feature gate: SigstoreImageVerification
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310607    4021 feature_gate.go:227] unrecognized feature gate: AdminNetworkPolicy
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310613    4021 feature_gate.go:227] unrecognized feature gate: AutomatedEtcdBackup
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310620    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProvider
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310626    4021 feature_gate.go:227] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310631    4021 feature_gate.go:227] unrecognized feature gate: DNSNameResolver
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310637    4021 feature_gate.go:227] unrecognized feature gate: RouteExternalCertificate
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310642    4021 feature_gate.go:227] unrecognized feature gate: VSphereStaticIPs
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310648    4021 feature_gate.go:227] unrecognized feature gate: GatewayAPI
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.310654    4021 feature_gate.go:250] feature gates: &{map[CloudDualStackNodeIPs:true DynamicResourceAllocation:false EventedPLEG:false MaxUnavailableStatefulSet:false NodeSwap:false RetroactiveDefaultStorageClass:false ValidatingAdmissionPolicy:false]}
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310744    4021 feature_gate.go:227] unrecognized feature gate: AdminNetworkPolicy
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310752    4021 feature_gate.go:227] unrecognized feature gate: AutomatedEtcdBackup
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310758    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProvider
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310763    4021 feature_gate.go:227] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310769    4021 feature_gate.go:227] unrecognized feature gate: DNSNameResolver
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310774    4021 feature_gate.go:227] unrecognized feature gate: RouteExternalCertificate
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310780    4021 feature_gate.go:227] unrecognized feature gate: VSphereStaticIPs
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310786    4021 feature_gate.go:227] unrecognized feature gate: GatewayAPI
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310791    4021 feature_gate.go:227] unrecognized feature gate: BuildCSIVolumes
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310797    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderAzure
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310803    4021 feature_gate.go:227] unrecognized feature gate: AzureWorkloadIdentity
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310811    4021 feature_gate.go:227] unrecognized feature gate: PrivateHostedZoneAWS
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310817    4021 feature_gate.go:227] unrecognized feature gate: AlibabaPlatform
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310822    4021 feature_gate.go:227] unrecognized feature gate: CSIDriverSharedResource
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310827    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderExternal
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310833    4021 feature_gate.go:227] unrecognized feature gate: ExternalCloudProviderGCP
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310838    4021 feature_gate.go:227] unrecognized feature gate: GCPLabelsTags
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310844    4021 feature_gate.go:227] unrecognized feature gate: InsightsConfigAPI
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310850    4021 feature_gate.go:227] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310859    4021 feature_gate.go:242] Setting GA feature gate RetroactiveDefaultStorageClass=false. It will be removed in a future release.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310865    4021 feature_gate.go:227] unrecognized feature gate: MachineAPIProviderOpenStack
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.310871    4021 feature_gate.go:227] unrecognized feature gate: SigstoreImageVerification
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.310877    4021 feature_gate.go:250] feature gates: &{map[CloudDualStackNodeIPs:true DynamicResourceAllocation:false EventedPLEG:false MaxUnavailableStatefulSet:false NodeSwap:false RetroactiveDefaultStorageClass:false ValidatingAdmissionPolicy:false]}
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.326947    4021 server.go:901] "Client rotation is on, will bootstrap in background"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.344795    4021 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.344850    4021 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.345122    4021 server.go:958] "Starting client certificate rotation"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.345136    4021 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Certificate rotation is enabled
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.346501    4021 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Certificate expiration is 2023-10-26 18:14:30 +0000 UTC, rotation deadline is 2023-10-26 12:50:08.748807282 +0000 UTC
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.346604    4021 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Waiting 17h51m45.402206794s for next certificate rotation
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.418803    4021 dynamic_cafile_content.go:119] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.418875    4021 util_unix.go:103] "Using this endpoint is deprecated, please consider using full URL format" endpoint="/var/run/crio/crio.sock" URL="unix:///var/run/crio/crio.sock"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.419583    4021 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.475627    4021 remote_runtime.go:143] "Validated CRI v1 runtime API"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.475742    4021 util_unix.go:103] "Using this endpoint is deprecated, please consider using full URL format" endpoint="/var/run/crio/crio.sock" URL="unix:///var/run/crio/crio.sock"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.549859    4021 remote_image.go:111] "Validated CRI v1 image API"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.565412    4021 manager.go:162] cAdvisor running in container: "/system.slice/kubelet.service"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.584380    4021 fs.go:133] Filesystem UUIDs: map[0568037f-73fb-422a-81f3-f4f96c744efb:/dev/vda4 3105-E089:/dev/vda2 ef506d8f-dd01-416d-a362-05e7d4dce2a3:/dev/vda3]
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.584395    4021 fs.go:134] Filesystem partitions: map[/dev/shm:{mountpoint:/dev/shm major:0 minor:23 fsType:tmpfs blockSize:0} /dev/vda3:{mountpoint:/boot major:252 minor:3 fsType:ext4 blockSize:0} /dev/vda4:{mountpoint:/var major:252 minor:4 fsType:xfs blockSize:0} /run:{mountpoint:/run major:0 minor:25 fsType:tmpfs blockSize:0} /sys/fs/cgroup:{mountpoint:/sys/fs/cgroup major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:44 fsType:tmpfs blockSize:0}]
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.610700    4021 manager.go:210] Machine: {Timestamp:2023-10-25 18:58:23.610203224 +0000 UTC m=+2.595158200 CPUVendorID:AuthenticAMD NumCores:8 NumPhysicalCores:1 NumSockets:8 CpuFrequency:2800000 MemoryCapacity:16759250944 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:4} {PageSize:2048 NumPages:0}] MachineID:d60d167946e84c789f1c90160e251f32 SystemUUID:d60d1679-46e8-4c78-9f1c-90160e251f32 BootID:f703f2ce-3884-4119-83f5-a24ce1d0aa20 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:23 Capacity:8379625472 Type:vfs Inodes:2045807 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:25 Capacity:3351851008 Type:vfs Inodes:819200 HasInodes:true} {Device:/sys/fs/cgroup DeviceMajor:0 DeviceMinor:26 Capacity:4194304 Type:vfs Inodes:1024 HasInodes:true} {Device:/dev/vda4 DeviceMajor:252 DeviceMinor:4 Capacity:42344624128 Type:vfs Inodes:20708800 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:44 Capacity:8379625472 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/vda3 DeviceMajor:252 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[252:0:{Name:vda Major:252 Minor:0 Size:42949672960 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:fa:16:3e:e6:c8:e6 Speed:0 Mtu:1342} {Name:br-int MacAddress:fe:3f:41:ee:62:9f Speed:0 Mtu:1242} {Name:enp3s0 MacAddress:fa:16:3e:e6:c8:e6 Speed:-1 Mtu:1342} {Name:enp4s0 MacAddress:fa:16:3e:73:67:7f Speed:-1 Mtu:1400} {Name:enp6s0 MacAddress:fa:16:3e:c2:2d:2a Speed:-1 Mtu:1500} {Name:genev_sys_6081 MacAddress:7a:5f:69:b0:08:20 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:fa:e3:a3:11:f1:d2 Speed:0 Mtu:1242} {Name:ovs-system MacAddress:4a:1d:52:68:8e:cf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:16759250944 HugePages:[{PageSize:1048576 NumPages:4} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:0 Size:16777216 Type:Unified Level:3}] SocketID:0} {Id:0 Threads:[1] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:1 Size:16777216 Type:Unified Level:3}] SocketID:1} {Id:0 Threads:[2] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:2 Size:16777216 Type:Unified Level:3}] SocketID:2} {Id:0 Threads:[3] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:3 Size:16777216 Type:Unified Level:3}] SocketID:3} {Id:0 Threads:[4] Caches:[{Id:4 Size:32768 Type:Data Level:1} {Id:4 Size:32768 Type:Instruction Level:1} {Id:4 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:4 Size:16777216 Type:Unified Level:3}] SocketID:4} {Id:0 Threads:[5] Caches:[{Id:5 Size:32768 Type:Data Level:1} {Id:5 Size:32768 Type:Instruction Level:1} {Id:5 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:5 Size:16777216 Type:Unified Level:3}] SocketID:5} {Id:0 Threads:[6] Caches:[{Id:6 Size:32768 Type:Data Level:1} {Id:6 Size:32768 Type:Instruction Level:1} {Id:6 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:6 Size:16777216 Type:Unified Level:3}] SocketID:6} {Id:0 Threads:[7] Caches:[{Id:7 Size:32768 Type:Data Level:1} {Id:7 Size:32768 Type:Instruction Level:1} {Id:7 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:7 Size:16777216 Type:Unified Level:3}] SocketID:7}] Caches:[] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.610833    4021 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.610872    4021 manager.go:226] Version: {KernelVersion:5.14.0-284.36.1.el9_2.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 415.92.202310241500-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.611008    4021 server.go:745] "Option --reserved-cpus is specified, it will overwrite the cpu setting in KubeReserved and SystemReserved" kubeReservedCPUs={"memory":"500Mi"} systemReservedCPUs={"cpu":"500m","ephemeral-storage":"1Gi","memory":"500Mi"}
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.611028    4021 server.go:753] "After cpu setting is overwritten" kubeReservedCPUs={"memory":"500Mi"} systemReservedCPUs={"cpu":"2","ephemeral-storage":"1Gi","memory":"500Mi"}
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.611352    4021 container_manager_linux.go:268] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.611499    4021 container_manager_linux.go:273] "Creating Container Manager object based on Node Config" nodeConfig={"RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":{"memory":"500Mi"},"SystemReserved":{"cpu":"2","ephemeral-storage":"1Gi","memory":"500Mi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"static","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":5000000000,"ExperimentalMemoryManagerPolicy":"None","ExperimentalMemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"best-effort","TopologyManagerPolicyOptions":null}
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.611526    4021 topology_manager.go:147] "Creating topology manager with policy per scope" topologyPolicyName="best-effort" topologyScopeName="container" topologyPolicyOptions={"PreferClosestNUMA":false}
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.611538    4021 container_manager_linux.go:304] "Creating device plugin manager"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.626423    4021 manager.go:135] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.626442    4021 server.go:66] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.626877    4021 cpu_manager.go:173] "Detected CPU topology" topology={"NumCPUs":8,"NumCores":8,"NumSockets":8,"NumNUMANodes":1,"CPUDetails":{"0":{"NUMANodeID":0,"SocketID":0,"CoreID":0},"1":{"NUMANodeID":0,"SocketID":1,"CoreID":1},"2":{"NUMANodeID":0,"SocketID":2,"CoreID":2},"3":{"NUMANodeID":0,"SocketID":3,"CoreID":3},"4":{"NUMANodeID":0,"SocketID":4,"CoreID":4},"5":{"NUMANodeID":0,"SocketID":5,"CoreID":5},"6":{"NUMANodeID":0,"SocketID":6,"CoreID":6},"7":{"NUMANodeID":0,"SocketID":7,"CoreID":7}}}
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.626916    4021 policy_static.go:139] "Static policy created with configuration" options={"FullPhysicalCPUsOnly":false,"DistributeCPUsAcrossNUMA":false,"AlignBySocket":false}
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.626937    4021 policy_static.go:175] "Reserved CPUs not available for exclusive assignment" reservedSize=2 reserved="0-1" reservedPhysicalCPUs="0-1"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.626945    4021 state_mem.go:36] "Initialized new in-memory state store"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.626991    4021 server.go:1203] "Using root directory" path="/var/lib/kubelet"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.628786    4021 kubelet.go:402] "Attempting to sync node with API server"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.628804    4021 kubelet.go:307] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.628820    4021 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.628832    4021 kubelet.go:318] "Adding apiserver pod source"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.628843    4021 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.682235    4021 kuberuntime_manager.go:257] "Container runtime initialized" containerRuntime="cri-o" version="1.28.1-7.rhaos4.15.gitd3d1f84.el9" apiVersion="v1"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.683399    4021 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-server-current.pem".
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.683966    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/azure-file"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.683981    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/vsphere-volume"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.684000    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.684010    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/rbd"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.684023    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699591    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699611    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699621    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699633    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699646    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699656    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/cephfs"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699667    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699676    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699687    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699697    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699706    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.699730    4021 plugins.go:635] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.700507    4021 server.go:1238] "Started kubelet"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn systemd[1]: Started Kubernetes Kubelet.
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.756684    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:23.756961    4021 kubelet.go:1444] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.758607    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:23.758731    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.758857    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:23.758940    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.756930    4021 server.go:162] "Starting to listen" address="10.0.3.42" port=10250
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.772843    4021 ratelimit.go:65] "Setting rate limiting for podresources endpoint" qps=100 burstTokens=10
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.774533    4021 server.go:462] "Adding debug handlers to kubelet server"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.775480    4021 server.go:233] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:23.779637    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.792165    4021 certificate_manager.go:356] kubernetes.io/kubelet-serving: Certificate rotation is enabled
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.792233    4021 certificate_manager.go:356] kubernetes.io/kubelet-serving: Certificate expiration is 2023-10-26 18:14:30 +0000 UTC, rotation deadline is 2023-10-26 15:38:48.470583603 +0000 UTC
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.792260    4021 certificate_manager.go:356] kubernetes.io/kubelet-serving: Waiting 20h40m24.678325331s for next certificate rotation
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.793395    4021 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.793943    4021 volume_manager.go:289] "The desired_state_of_world populator starts"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.794030    4021 volume_manager.go:291] "Starting Kubelet Volume Manager"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.794499    4021 desired_state_of_world_populator.go:151] "Desired state populator starts to run"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:23.800380    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:23.800519    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:23.800638    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="200ms"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.810418    4021 factory.go:153] Registering CRI-O factory
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.810428    4021 factory.go:55] Registering systemd factory
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.810504    4021 factory.go:103] Registering Raw factory
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.811127    4021 manager.go:1186] Started watching for new ooms in manager
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.813384    4021 manager.go:299] Starting recovery of all containers
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.824805    4021 manager.go:304] Recovery completed
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907173    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce8d94be-2c75-42a5-a8e1-60275cefdcbd" volumeName="kubernetes.io/configmap/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-metrics-client-ca" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907200    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce8d94be-2c75-42a5-a8e1-60275cefdcbd" volumeName="kubernetes.io/empty-dir/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-textfile" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907212    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="de1fef77-9a99-40d0-a88a-9a1d851e832f" volumeName="kubernetes.io/projected/de1fef77-9a99-40d0-a88a-9a1d851e832f-kube-api-access-5tbfh" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907222    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="38bac755-4b7a-447a-bea0-6c1be8066cd8" volumeName="kubernetes.io/secret/38bac755-4b7a-447a-bea0-6c1be8066cd8-metrics-certs" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907233    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3" volumeName="kubernetes.io/configmap/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-var-lib-tuned-profiles-data" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907242    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="93d972c7-c2ab-4f1e-8b56-d8d40be93cb2" volumeName="kubernetes.io/configmap/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-multus-daemon-config" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907252    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce8d94be-2c75-42a5-a8e1-60275cefdcbd" volumeName="kubernetes.io/secret/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-tls" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907263    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="dbed798e-c84c-4287-b7e4-aefe4a08c3cb" volumeName="kubernetes.io/secret/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-metrics-tls" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907275    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="60df7285-d18c-4337-9e6b-27c667f60eee" volumeName="kubernetes.io/configmap/60df7285-d18c-4337-9e6b-27c667f60eee-cacert" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907285    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a75dddb-d3b3-408f-9cab-46b51953ccc8" volumeName="kubernetes.io/projected/6a75dddb-d3b3-408f-9cab-46b51953ccc8-kube-api-access-6rqmd" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907296    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a75dddb-d3b3-408f-9cab-46b51953ccc8" volumeName="kubernetes.io/secret/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907306    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="9954521a-e814-4b76-b21a-3b08ad936c10" volumeName="kubernetes.io/projected/9954521a-e814-4b76-b21a-3b08ad936c10-kube-api-access-jzd52" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907316    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="aab63cd9-e2d8-4dff-8d13-22cb7a673703" volumeName="kubernetes.io/projected/aab63cd9-e2d8-4dff-8d13-22cb7a673703-kube-api-access-8tzhz" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907327    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce1d733d-5d0c-42c3-93bf-254808356343" volumeName="kubernetes.io/projected/ce1d733d-5d0c-42c3-93bf-254808356343-kube-api-access-8ntwx" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907337    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce8d94be-2c75-42a5-a8e1-60275cefdcbd" volumeName="kubernetes.io/projected/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-kube-api-access-vkjd6" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907347    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf0522a-ef9e-45e0-bee2-397d22cc2f3b" volumeName="kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-config-cinderplugin" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907358    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3" volumeName="kubernetes.io/projected/39ff7e3b-ff7a-4e89-bbf1-c0cedc0d1cd3-kube-api-access-9zszm" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907368    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06" volumeName="kubernetes.io/projected/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-kube-api-access-8jgbw" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907377    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="93d972c7-c2ab-4f1e-8b56-d8d40be93cb2" volumeName="kubernetes.io/configmap/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-cni-binary-copy" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907388    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf0522a-ef9e-45e0-bee2-397d22cc2f3b" volumeName="kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-cacert" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907398    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="38bac755-4b7a-447a-bea0-6c1be8066cd8" volumeName="kubernetes.io/projected/38bac755-4b7a-447a-bea0-6c1be8066cd8-kube-api-access-kvsgx" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907408    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06" volumeName="kubernetes.io/configmap/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907418    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf0522a-ef9e-45e0-bee2-397d22cc2f3b" volumeName="kubernetes.io/projected/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-kube-api-access-5smgg" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907427    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="60df7285-d18c-4337-9e6b-27c667f60eee" volumeName="kubernetes.io/configmap/60df7285-d18c-4337-9e6b-27c667f60eee-non-standard-root-system-trust-ca-bundle" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907438    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a75dddb-d3b3-408f-9cab-46b51953ccc8" volumeName="kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-ca" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907447    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef062c82f679a1ef42be9dd35e115e2f" volumeName="kubernetes.io/empty-dir/ef062c82f679a1ef42be9dd35e115e2f-run-dir" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907460    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf0522a-ef9e-45e0-bee2-397d22cc2f3b" volumeName="kubernetes.io/configmap/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-non-standard-root-system-trust-ca-bundle" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907472    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="60df7285-d18c-4337-9e6b-27c667f60eee" volumeName="kubernetes.io/projected/60df7285-d18c-4337-9e6b-27c667f60eee-kube-api-access-h8ncd" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907483    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a75dddb-d3b3-408f-9cab-46b51953ccc8" volumeName="kubernetes.io/secret/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovn-cert" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907494    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="9954521a-e814-4b76-b21a-3b08ad936c10" volumeName="kubernetes.io/configmap/9954521a-e814-4b76-b21a-3b08ad936c10-serviceca" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907506    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="93d972c7-c2ab-4f1e-8b56-d8d40be93cb2" volumeName="kubernetes.io/projected/93d972c7-c2ab-4f1e-8b56-d8d40be93cb2-kube-api-access-xf5gb" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907516    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="aab63cd9-e2d8-4dff-8d13-22cb7a673703" volumeName="kubernetes.io/configmap/aab63cd9-e2d8-4dff-8d13-22cb7a673703-mcd-auth-proxy-config" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907527    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="aab63cd9-e2d8-4dff-8d13-22cb7a673703" volumeName="kubernetes.io/secret/aab63cd9-e2d8-4dff-8d13-22cb7a673703-proxy-tls" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907538    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce8d94be-2c75-42a5-a8e1-60275cefdcbd" volumeName="kubernetes.io/secret/ce8d94be-2c75-42a5-a8e1-60275cefdcbd-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907549    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="dbed798e-c84c-4287-b7e4-aefe4a08c3cb" volumeName="kubernetes.io/configmap/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-config-volume" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907559    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="21b6dd11-c00e-4ec7-a192-703f8a824b80" volumeName="kubernetes.io/projected/21b6dd11-c00e-4ec7-a192-703f8a824b80-kube-api-access-g6vvr" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907569    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a75dddb-d3b3-408f-9cab-46b51953ccc8" volumeName="kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-ovnkube-config" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907578    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06" volumeName="kubernetes.io/configmap/75b8ab9f-55bc-4837-ace0-e5ca7b7bfa06-cni-binary-copy" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907588    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdf0522a-ef9e-45e0-bee2-397d22cc2f3b" volumeName="kubernetes.io/secret/fdf0522a-ef9e-45e0-bee2-397d22cc2f3b-secret-cinderplugin" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907598    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f285036-37bd-4419-a167-e41819a56c33" volumeName="kubernetes.io/projected/2f285036-37bd-4419-a167-e41819a56c33-kube-api-access-n4n5q" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907608    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a75dddb-d3b3-408f-9cab-46b51953ccc8" volumeName="kubernetes.io/configmap/6a75dddb-d3b3-408f-9cab-46b51953ccc8-env-overrides" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907621    4021 reconstruct_new.go:135] "Volume is marked as uncertain and added into the actual state" pod="" podName="dbed798e-c84c-4287-b7e4-aefe4a08c3cb" volumeName="kubernetes.io/projected/dbed798e-c84c-4287-b7e4-aefe4a08c3cb-kube-api-access-2466z" seLinuxMountContext=""
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907628    4021 reconstruct_new.go:102] "Volume reconstruction finished"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.907634    4021 reconciler_new.go:29] "Reconciler: start to sync state"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.908788    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.925323    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.925348    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.925357    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.925382    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.932323    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.933336    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.933349    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.933357    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.933733    4021 cpu_manager.go:215] "Starting CPU manager" policy="static"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.933742    4021 cpu_manager.go:216] "Reconciling" reconcilePeriod="5s"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.933771    4021 state_mem.go:36] "Initialized new in-memory state store"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:23.959620    4021 state_mem.go:88] "Updated default CPUSet" cpuSet="0-7"
Oct 25 18:58:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:23.977767    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.001535    4021 memory_manager.go:169] "Starting memorymanager" policy="None"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.001576    4021 state_mem.go:35] "Initializing new in-memory state store"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:24.005769    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="400ms"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.091740    4021 manager.go:295] "Starting Device Plugin manager"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.091780    4021 manager.go:471] "Failed to read data from checkpoint" checkpoint="kubelet_internal_checkpoint" err="checkpoint is not found"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.091788    4021 server.go:79] "Starting device plugin registration server"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.092081    4021 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.092122    4021 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.092127    4021 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:24.095301    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.178070    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.178809    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.178900    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.178969    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.179065    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:24.182947    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.214333    4021 kubelet_network_linux.go:50] "Initialized iptables rules." protocol="IPv4"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.217026    4021 kubelet_network_linux.go:50] "Initialized iptables rules." protocol="IPv6"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.217054    4021 status_manager.go:217] "Starting to sync pod status with apiserver"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.217073    4021 kubelet.go:2313] "Starting kubelet main sync loop"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:24.217167    4021 kubelet.go:2337] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:24.222611    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:24.222718    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317423    4021 kubelet.go:2399] "SyncLoop ADD" source="file" pods=["openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn","openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"]
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317475    4021 topology_manager.go:215] "Topology Admit Handler" podUID="74b8c21149c0164c4da6a5858bca8c45" podNamespace="openshift-openstack-infra" podName="coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317573    4021 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-coredns"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317592    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-coredns"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317599    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-coredns"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317611    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317617    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317622    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317637    4021 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317644    4021 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-coredns"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317650    4021 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-coredns"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317665    4021 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317673    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317680    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317685    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317690    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317695    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317700    4021 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317706    4021 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317712    4021 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317722    4021 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns-monitor"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317730    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns-monitor"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317737    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns-monitor"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317742    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317747    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317751    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317757    4021 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317762    4021 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns-monitor"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317768    4021 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="coredns-monitor"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.317790    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319624    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319642    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319649    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319716    4021 topology_manager.go:215] "Topology Admit Handler" podUID="ef062c82f679a1ef42be9dd35e115e2f" podNamespace="openshift-openstack-infra" podName="keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319727    4021 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-keepalived"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319738    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-keepalived"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319744    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-keepalived"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319750    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319755    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319760    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319767    4021 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319773    4021 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-keepalived"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319779    4021 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="render-config-keepalived"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319792    4021 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319799    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319805    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319810    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319815    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319820    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319827    4021 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319834    4021 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319842    4021 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319862    4021 scope_container.go:75] "TopologyHints" hints={} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived-monitor"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319873    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived-monitor"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319902    4021 scope_container.go:75] "TopologyHints" hints=null pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived-monitor"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319910    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319916    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319922    4021 policy.go:71] "Hint Provider has no preference for NUMA affinity with any resource"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319930    4021 scope_container.go:83] "ContainerTopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true}
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319937    4021 scope_container.go:50] "Best TopologyHint" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived-monitor"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319946    4021 scope_container.go:56] "Topology Affinity" bestHint={"NUMANodeAffinity":1,"Preferred":true} pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" containerName="keepalived-monitor"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.319967    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.320238    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.320731    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.320747    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.320757    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.320852    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.321095    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.321118    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.321127    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.321476    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.321570    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.321810    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:24.409853    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="800ms"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.469029    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"run-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef062c82f679a1ef42be9dd35e115e2f-run-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.494638    4021 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-resource-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.494724    4021 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-kubeconfig\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.494801    4021 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfig\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.494889    4021 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-conf-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.494961    4021 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfigvarlib\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfigvarlib\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.495027    4021 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"chroot-host\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-chroot-host\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.495092    4021 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nodeip-configuration\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-nodeip-configuration\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.495162    4021 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-conf-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.495228    4021 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nm-resolv\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-nm-resolv\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.495292    4021 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-resource-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.495356    4021 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"script-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-script-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.518274    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"run-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef062c82f679a1ef42be9dd35e115e2f-run-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.583394    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.583933    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.584027    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.584093    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.584163    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:24.590598    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.600925    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-conf-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601035    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"nm-resolv\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-nm-resolv\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601136    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-resource-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601257    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"script-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-script-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601037    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-conf-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601066    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"nm-resolv\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-nm-resolv\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601227    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-resource-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601351    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"script-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-script-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601666    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kubeconfigvarlib\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfigvarlib\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601783    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"chroot-host\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-chroot-host\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601915    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"nodeip-configuration\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-nodeip-configuration\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.602051    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-resource-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.602169    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-kubeconfig\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601753    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kubeconfigvarlib\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfigvarlib\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601881    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"chroot-host\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-chroot-host\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.601953    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"nodeip-configuration\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-nodeip-configuration\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.602139    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-resource-dir\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.602265    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/74b8c21149c0164c4da6a5858bca8c45-kubeconfig\") pod \"coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"74b8c21149c0164c4da6a5858bca8c45\") " pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.602641    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfig\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.602717    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-kubeconfig\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.602804    4021 reconciler_common.go:231] "operationExecutor.MountVolume started for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-conf-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.602897    4021 operation_generator.go:718] "MountVolume.SetUp succeeded for volume \"conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef062c82f679a1ef42be9dd35e115e2f-conf-dir\") pod \"keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" (UID: \"ef062c82f679a1ef42be9dd35e115e2f\") " pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:24.631227    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:24.631343    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.645562    4021 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.645875    4021 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:24.761251    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:24.784828    4021 manager.go:1159] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b WatchSource:0}: Error finding container e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b: Status 404 returned error can't find the container with id e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:24.844115    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:24.844175    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:25.120711    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:25.120763    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:25.213736    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="1.6s"
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:25.221930    4021 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event={"ID":"ef062c82f679a1ef42be9dd35e115e2f","Type":"ContainerStarted","Data":"e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"}
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:25.222556    4021 kubelet.go:2431] "SyncLoop (PLEG): event for pod" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event={"ID":"74b8c21149c0164c4da6a5858bca8c45","Type":"ContainerStarted","Data":"d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"}
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:25.391768    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:25.392292    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:25.392318    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:25.392329    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:25.392347    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:25.396601    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:25.652248    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:25.652280    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:25.762821    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:25.992604    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:58:25Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d5e3213c7f7c688288ade97c8c025f8bb0bd91bea8cc1690d844666bc26a6569.scope/cpuset.cpus: permission denied"
Oct 25 18:58:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.014440    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T18:58:25Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d5e3213c7f7c688288ade97c8c025f8bb0bd91bea8cc1690d844666bc26a6569.scope/cpuset.cpus: permission denied"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.014496    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:58:25Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d5e3213c7f7c688288ade97c8c025f8bb0bd91bea8cc1690d844666bc26a6569.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.016123    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:58:25Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-210639c9458300807f6dfd70abbb220dbef2fc7a1bb8ab4502e5d0363041ce64.scope/cpuset.cpus: permission denied"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.016180    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T18:58:25Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-210639c9458300807f6dfd70abbb220dbef2fc7a1bb8ab4502e5d0363041ce64.scope/cpuset.cpus: permission denied"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.016220    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:58:25Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-210639c9458300807f6dfd70abbb220dbef2fc7a1bb8ab4502e5d0363041ce64.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.223098    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.223382    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.223805    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.223824    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.223834    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.223865    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.223880    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.223888    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.398594    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:58:26Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a9fcb682c5e802e7ec9104f07d527c6819cea4598223ad83bb506dd71de5ecec.scope/cpuset.cpus: permission denied"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.398718    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T18:58:26Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a9fcb682c5e802e7ec9104f07d527c6819cea4598223ad83bb506dd71de5ecec.scope/cpuset.cpus: permission denied"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.398764    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:58:26Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a9fcb682c5e802e7ec9104f07d527c6819cea4598223ad83bb506dd71de5ecec.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.421640    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:58:26Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e3fd89444003163f19c3a90898eebe2f7eea4df94a1c9ce82e27ebc4cff5cca9.scope/cpuset.cpus: permission denied"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.421735    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T18:58:26Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e3fd89444003163f19c3a90898eebe2f7eea4df94a1c9ce82e27ebc4cff5cca9.scope/cpuset.cpus: permission denied"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.421805    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:58:26Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e3fd89444003163f19c3a90898eebe2f7eea4df94a1c9ce82e27ebc4cff5cca9.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:26.667795    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.667840    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.760834    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.821125    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="3.2s"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:26.928534    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:26.928584    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.996834    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.997545    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.997564    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.997575    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:26.997593    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:27.001539    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:27.503360    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:27.503689    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:27.691858    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:27.691891    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:27.764383    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:28.762669    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:29.762780    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:30.026981    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="6.4s"
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:30.202373    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:30.203087    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:30.203106    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:30.203116    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:30.203138    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:30.208254    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:30.667881    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:30.668010    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:30.668144    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:30.668228    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:30.761893    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:30.773763    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:58:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:31.763244    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:32.500293    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:32.500610    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:32.520537    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:32.520634    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:32.753311    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:32.753423    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:32.762438    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:33.671253    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:33.671629    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:58:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:33.764616    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:34.095471    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:58:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:34.761320    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:35.151265    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:35.151294    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:58:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:35.761402    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:36.431990    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:58:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:36.608559    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:36.609271    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:36.609357    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:36.609425    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:36.609500    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:36.615402    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:36.762114    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:37.218131    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:37.218847    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:37.218937    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:37.219052    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:37.333566    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:58:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:58:37Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-41c4fd99292d48ca21624afa1dd45b6debf2ea31d98b784ad6c5c5109fc8d60e.scope/cpuset.cpus: permission denied"
Oct 25 18:58:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 18:58:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:37.333664    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T18:58:37Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-41c4fd99292d48ca21624afa1dd45b6debf2ea31d98b784ad6c5c5109fc8d60e.scope/cpuset.cpus: permission denied"
Oct 25 18:58:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:37.333716    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:58:37Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-41c4fd99292d48ca21624afa1dd45b6debf2ea31d98b784ad6c5c5109fc8d60e.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 18:58:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:37.761693    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:38.763977    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:39.758409    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:40.217939    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:40.218677    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:40.218764    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:40.218848    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:40.341571    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:58:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:58:40Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-796d587c4b4bfdcc2037637125c66c84e164feeab24a3434b1d4189b02df8ca8.scope/cpuset.cpus: permission denied"
Oct 25 18:58:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 18:58:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:40.341671    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T18:58:40Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-796d587c4b4bfdcc2037637125c66c84e164feeab24a3434b1d4189b02df8ca8.scope/cpuset.cpus: permission denied"
Oct 25 18:58:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:40.341720    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:58:40Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-796d587c4b4bfdcc2037637125c66c84e164feeab24a3434b1d4189b02df8ca8.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 18:58:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:40.764278    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:40.778142    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:58:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:41.764662    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:42.704110    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:42.704171    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:42.761861    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:43.097930    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:43.097980    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:43.440817    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:43.616311    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:43.616811    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:43.616933    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:43.617039    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:43.617188    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:43.620958    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:43.763341    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:43.814426    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:43.814532    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:44.017459    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:44.017575    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:44.095967    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:58:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:58:44.641497    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:44.641529    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:44.761043    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:44.898344    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:44.898463    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:58:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:45.761314    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:46.761522    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:47.762487    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:48.762118    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:49.765092    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:50.217933    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:50.218641    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:50.218721    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:50.218785    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:50.333383    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:58:50Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-3deabe9b305732dadb122e8bbf3b3e91671bbad8ab2c291b15b0b1a21157a257.scope/cpuset.cpus: permission denied"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:50.333492    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T18:58:50Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-3deabe9b305732dadb122e8bbf3b3e91671bbad8ab2c291b15b0b1a21157a257.scope/cpuset.cpus: permission denied"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:50.333546    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:58:50Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-3deabe9b305732dadb122e8bbf3b3e91671bbad8ab2c291b15b0b1a21157a257.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:50.445442    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:50.622050    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:50.622757    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:50.622772    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:50.622780    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:50.622813    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:50.628539    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:50.764276    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:50.781680    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:58:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:51.760438    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:52.199899    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:52.199947    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:58:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:52.762810    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:53.760786    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:54.096459    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:58:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:54.762400    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:55.217788    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:55.218475    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:55.218566    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:55.218630    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:55.348158    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:58:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:58:55Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d25d1b422f01b9bcf732e3665cdfa41776f5cf93a55aea33aed3c153758743ce.scope/cpuset.cpus: permission denied"
Oct 25 18:58:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 18:58:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:55.348253    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T18:58:55Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d25d1b422f01b9bcf732e3665cdfa41776f5cf93a55aea33aed3c153758743ce.scope/cpuset.cpus: permission denied"
Oct 25 18:58:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:55.348306    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:58:55Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d25d1b422f01b9bcf732e3665cdfa41776f5cf93a55aea33aed3c153758743ce.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 18:58:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:55.761804    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:56.762001    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:57.452534    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:58:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:57.629060    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:58:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:57.629610    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:58:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:57.629712    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:58:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:57.629802    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:58:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:57.629920    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:58:57.633998    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:58:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:57.762968    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:58.761407    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:58:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:58:59.762763    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:00.625775    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:00.625851    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:00.762453    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:00.789187    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:59:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:01.760257    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:02.218066    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:02.218612    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:02.218636    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:02.218647    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:02.340572    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:59:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:59:02Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0dc94ab56b98094bf391baade23cdb238e5067f678d48ecbcca4718aeca311af.scope/cpuset.cpus: permission denied"
Oct 25 18:59:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 18:59:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:02.340677    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T18:59:02Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0dc94ab56b98094bf391baade23cdb238e5067f678d48ecbcca4718aeca311af.scope/cpuset.cpus: permission denied"
Oct 25 18:59:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:02.340730    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:59:02Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0dc94ab56b98094bf391baade23cdb238e5067f678d48ecbcca4718aeca311af.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 18:59:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:02.761266    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:03.669612    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:03.670064    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:03.761545    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:04.096925    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:04.456673    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:59:04.573515    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:04.573633    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:04.634188    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:04.634776    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:04.634861    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:04.634943    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:04.635019    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:04.637684    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:59:04.637794    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:04.637898    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:04.761906    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:05.150608    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:05.150636    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:05.761767    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:06.761901    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:59:07.437542    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:07.437603    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:07.760572    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:08.762864    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:59:09.119317    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:09.119371    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:09.763495    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:10.217784    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:10.218302    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:10.218402    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:10.218478    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:10.336519    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:59:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:59:10Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e0c60577b4d6d9dad01f0888ced363dcdcd54dd3cd94ee56b2a28b9d99f593cf.scope/cpuset.cpus: permission denied"
Oct 25 18:59:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 18:59:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:10.336608    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T18:59:10Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e0c60577b4d6d9dad01f0888ced363dcdcd54dd3cd94ee56b2a28b9d99f593cf.scope/cpuset.cpus: permission denied"
Oct 25 18:59:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:10.336653    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:59:10Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e0c60577b4d6d9dad01f0888ced363dcdcd54dd3cd94ee56b2a28b9d99f593cf.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 18:59:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:10.762675    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:10.794575    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:59:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:11.461990    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:59:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:11.637965    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:11.638556    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:11.638690    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:11.638791    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:11.638902    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:11.645186    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:11.760808    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:12.762328    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:13.098722    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:13.098753    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:13.757323    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:14.097996    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:59:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:14.765278    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:14.895553    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:14.895600    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:15.218008    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:15.218840    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:15.218944    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:15.219021    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:15.333546    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:59:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:59:15Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-1606c43420b6caa84d5a1c6fc86fff86c3ac1296c97f362682fcb0578c0525f1.scope/cpuset.cpus: permission denied"
Oct 25 18:59:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 18:59:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:15.333653    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T18:59:15Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-1606c43420b6caa84d5a1c6fc86fff86c3ac1296c97f362682fcb0578c0525f1.scope/cpuset.cpus: permission denied"
Oct 25 18:59:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:15.333710    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:59:15Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-1606c43420b6caa84d5a1c6fc86fff86c3ac1296c97f362682fcb0578c0525f1.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 18:59:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:15.761984    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:16.762326    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:17.762010    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:18.468318    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:59:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:18.645243    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:18.645812    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:18.645977    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:18.646096    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:18.646187    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:18.653520    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:18.761976    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:19.760878    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:20.761260    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:20.799748    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:59:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:21.761248    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:22.198675    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:22.198730    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:22.217921    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:22.218453    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:22.218471    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:22.218481    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:22.367444    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:59:22Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-938c7315cdfd4dd7a9dc26584bafae1e8f6aa000346068109c61d60bf524fdc0.scope/cpuset.cpus: permission denied"
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:22.367535    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T18:59:22Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-938c7315cdfd4dd7a9dc26584bafae1e8f6aa000346068109c61d60bf524fdc0.scope/cpuset.cpus: permission denied"
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:22.367581    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:59:22Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-938c7315cdfd4dd7a9dc26584bafae1e8f6aa000346068109c61d60bf524fdc0.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 18:59:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:22.761513    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:23.762092    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:23.794380    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 18:59:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:23.794412    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 18:59:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:24.098075    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:59:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:24.762023    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:25.473195    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:59:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:25.653909    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:25.654517    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:25.654542    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:25.654552    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:25.654592    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:25.696630    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:25.761938    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:26.765271    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:27.762886    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:28.762247    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:29.770348    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:30.218021    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:30.218961    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:30.219062    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:30.219143    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:30.348932    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:59:30Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-53fb690ee2973219b4224c8c04829ea66dac385cab9254951223f747bf82fb73.scope/cpuset.cpus: permission denied"
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:30.349044    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T18:59:30Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-53fb690ee2973219b4224c8c04829ea66dac385cab9254951223f747bf82fb73.scope/cpuset.cpus: permission denied"
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:30.349102    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:59:30Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-53fb690ee2973219b4224c8c04829ea66dac385cab9254951223f747bf82fb73.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:30.692443    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:30.692479    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:30.762353    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:30.803616    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:59:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:31.761011    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:32.476971    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:59:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:32.697110    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:32.698020    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:32.698140    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:32.698240    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:32.698329    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:32.702216    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:32.761139    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:33.671763    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:33.672189    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:33.760922    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:34.099081    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:59:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:34.217382    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:34.218014    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:34.218118    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:34.218207    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:34.344570    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:59:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:59:34Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-2b1f01a3629c6cdf7c52d10fd76adfb97d635e6f3f1a0412b821256f853c9202.scope/cpuset.cpus: permission denied"
Oct 25 18:59:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 18:59:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:34.344745    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T18:59:34Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-2b1f01a3629c6cdf7c52d10fd76adfb97d635e6f3f1a0412b821256f853c9202.scope/cpuset.cpus: permission denied"
Oct 25 18:59:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:34.344799    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:59:34Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-2b1f01a3629c6cdf7c52d10fd76adfb97d635e6f3f1a0412b821256f853c9202.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 18:59:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:34.761891    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:35.157631    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:35.157800    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:59:35.376944    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:35.377085    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:35.761618    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:36.762575    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:59:36.768214    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:36.768350    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:59:37.121808    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:37.121958    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:37.761486    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:38.761948    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:39.481325    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:59:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:39.702840    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:39.703523    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:39.703627    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:39.703715    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:39.703805    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:39.710657    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:39.761461    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:40.761759    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:40.810033    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:59:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:41.764666    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:42.761474    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:43.097270    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:43.097301    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:43.762566    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:44.099293    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:44.218171    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:44.218678    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:44.218764    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:44.218851    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:44.344574    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:59:44Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-5b828fd7a1d100f37441a0f8a5aa0d0b82d483a240f23d389d137edbdb2f5529.scope/cpuset.cpus: permission denied"
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:44.344659    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T18:59:44Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-5b828fd7a1d100f37441a0f8a5aa0d0b82d483a240f23d389d137edbdb2f5529.scope/cpuset.cpus: permission denied"
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:44.344709    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:59:44Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-5b828fd7a1d100f37441a0f8a5aa0d0b82d483a240f23d389d137edbdb2f5529.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:44.760936    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:44.892488    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:44.892519    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:45.217534    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:45.218106    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:45.218125    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:45.218133    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:45.332868    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:59:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:59:45Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-33b7387816a596cb08e908508297983296b49465f6f06ca83f3510d961d46ab5.scope/cpuset.cpus: permission denied"
Oct 25 18:59:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 18:59:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:45.332990    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T18:59:45Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-33b7387816a596cb08e908508297983296b49465f6f06ca83f3510d961d46ab5.scope/cpuset.cpus: permission denied"
Oct 25 18:59:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:45.333037    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:59:45Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-33b7387816a596cb08e908508297983296b49465f6f06ca83f3510d961d46ab5.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 18:59:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:45.761258    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:46.486481    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:59:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:46.710906    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:46.711451    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:46.711490    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:46.711500    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:46.711517    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:46.716742    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:46.765975    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:47.761952    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 18:59:47.934322    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:47.934392    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:48.761225    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:49.761677    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:50.761347    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:50.814558    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 18:59:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:51.761114    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:52.198042    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:52.198492    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 18:59:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:52.762100    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:53.493391    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 18:59:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:53.716840    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:53.717798    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:53.717853    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:53.717868    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:53.717888    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:53.722424    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 18:59:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:53.761431    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:54.099991    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 18:59:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:54.764962    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:55.761911    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:56.761274    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:57.763102    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:58.762612    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:59.218174    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:59.218315    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:59.218954    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:59.219052    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:59.219095    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:59.219231    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:59.219246    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:59.219256    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:59.402812    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:59:59Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-0e0e22506f318c1bb09def72bcd7685f7e0eadf6bb0d01eff52e68da1c88a8aa.scope/cpuset.cpus: permission denied"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:59.402953    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T18:59:59Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-0e0e22506f318c1bb09def72bcd7685f7e0eadf6bb0d01eff52e68da1c88a8aa.scope/cpuset.cpus: permission denied"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:59.403027    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:59:59Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-0e0e22506f318c1bb09def72bcd7685f7e0eadf6bb0d01eff52e68da1c88a8aa.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:59.421295    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T18:59:59Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e6256fe8bf86d35c8e27e62c22981493a190fa86d166a79a5085f889a7379535.scope/cpuset.cpus: permission denied"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:59.421359    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T18:59:59Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e6256fe8bf86d35c8e27e62c22981493a190fa86d166a79a5085f889a7379535.scope/cpuset.cpus: permission denied"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 18:59:59.421403    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T18:59:59Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e6256fe8bf86d35c8e27e62c22981493a190fa86d166a79a5085f889a7379535.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 18:59:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 18:59:59.761459    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:00.502687    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:00:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:00.633421    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:00.633467    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:00.722842    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:00.723399    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:00.723504    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:00.723580    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:00.723665    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:00.733902    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:00.768971    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:00.821056    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:00:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:01.767073    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:02.765753    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:03.673066    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:03.673380    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:03.764451    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:04.100544    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:00:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:04.767150    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:05.150348    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:05.150555    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:05.760846    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:06.762123    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:07.507190    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:00:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:07.734290    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:07.734918    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:07.735015    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:07.735086    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:07.735166    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:07.739305    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:07.760520    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:08.762389    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:09.760796    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:10.764165    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:10.825713    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:00:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:10.825914    4021 event.go:228] Unable to write event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebf9a36df9a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 700483994, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}' (retry limit exceeded!)
Oct 25 19:00:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:10.830364    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:11.133315    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:00:11.154504    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:11.154639    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:11.217547    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:11.218172    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:11.218277    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:11.218368    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:11.339688    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:00:11Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-94297759f8f6397e1ea438535170541b7646d44736ca3e848a02d2fdec556a52.scope/cpuset.cpus: permission denied"
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:11.339789    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:00:11Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-94297759f8f6397e1ea438535170541b7646d44736ca3e848a02d2fdec556a52.scope/cpuset.cpus: permission denied"
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:11.339838    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:00:11Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-94297759f8f6397e1ea438535170541b7646d44736ca3e848a02d2fdec556a52.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:00:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:11.761685    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:12.761026    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:13.096342    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:13.096373    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:13.217607    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:13.218319    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:13.218405    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:13.218478    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:00:13.309341    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-2a7193a80f404bf7186df267cbf9107f65e86336e0a843db0b9cfe0ceeef92ec.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-2a7193a80f404bf7186df267cbf9107f65e86336e0a843db0b9cfe0ceeef92ec.scope: no such file or directory
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:13.336513    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:00:13Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-2a7193a80f404bf7186df267cbf9107f65e86336e0a843db0b9cfe0ceeef92ec.scope/cpuset.cpus: permission denied"
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:13.336641    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:00:13Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-2a7193a80f404bf7186df267cbf9107f65e86336e0a843db0b9cfe0ceeef92ec.scope/cpuset.cpus: permission denied"
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:13.336689    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:00:13Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-2a7193a80f404bf7186df267cbf9107f65e86336e0a843db0b9cfe0ceeef92ec.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:00:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:13.761246    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:14.100868    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:00:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:14.511660    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:00:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:14.739906    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:14.740503    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:14.740526    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:14.740537    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:14.740573    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:14.744670    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:14.761429    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:14.891927    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:14.891958    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:15.761676    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:16.762023    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:17.761657    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:00:18.304863    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:18.305185    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:18.761548    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:19.758458    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:20.766406    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:21.139247    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:00:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:21.518584    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:00:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:21.745582    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:21.746321    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:21.746341    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:21.746353    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:21.746372    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:21.751152    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:21.761306    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:22.197802    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:22.197847    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:22.761124    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:00:23.416032    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:23.416090    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:23.760920    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:23.795057    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:00:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:23.795170    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:00:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:24.101504    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:00:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:24.761524    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:25.217368    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:25.218058    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:25.218151    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:25.218272    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:25.344570    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:00:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:00:25Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-9372cc32018fe94d7538bfa9884230ef96936859b67b1d8c0f21ed09cd5ce394.scope/cpuset.cpus: permission denied"
Oct 25 19:00:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:00:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:25.344671    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:00:25Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-9372cc32018fe94d7538bfa9884230ef96936859b67b1d8c0f21ed09cd5ce394.scope/cpuset.cpus: permission denied"
Oct 25 19:00:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:25.344724    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:00:25Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-9372cc32018fe94d7538bfa9884230ef96936859b67b1d8c0f21ed09cd5ce394.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:00:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:25.766742    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:26.217844    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:26.218424    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:26.218452    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:26.218464    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:00:26.308489    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-5eb9266893312336ed3c81fd349f9d058ba64b1b610a1ec53a544b92d28f5d2f.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-5eb9266893312336ed3c81fd349f9d058ba64b1b610a1ec53a544b92d28f5d2f.scope: no such file or directory
Oct 25 19:00:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:26.346333    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:00:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:00:26Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-5eb9266893312336ed3c81fd349f9d058ba64b1b610a1ec53a544b92d28f5d2f.scope/cpuset.cpus: permission denied"
Oct 25 19:00:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:00:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:26.346428    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:00:26Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-5eb9266893312336ed3c81fd349f9d058ba64b1b610a1ec53a544b92d28f5d2f.scope/cpuset.cpus: permission denied"
Oct 25 19:00:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:26.346476    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:00:26Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-5eb9266893312336ed3c81fd349f9d058ba64b1b610a1ec53a544b92d28f5d2f.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:00:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:26.765159    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:27.761775    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:28.523877    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:00:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:00:28.595035    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:28.595086    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:28.751581    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:28.752122    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:28.752237    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:28.752330    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:28.752425    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:28.756455    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:28.760928    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:29.760796    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:30.627001    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:30.627200    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:30.760650    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:31.145032    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:00:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:31.763463    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:32.764684    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:33.668844    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:33.668881    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:33.762151    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:34.102282    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:00:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:34.763233    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:35.151695    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:35.151725    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:35.529865    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:00:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:35.757573    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:35.758377    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:35.758530    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:35.758632    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:35.758726    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:35.762420    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:35.762425    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:36.217423    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:36.218104    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:36.218203    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:36.218273    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:36.347607    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:00:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:00:36Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-db35cc655287784ee53cf7183b6a22602c1023901723750976084969b92af8e9.scope/cpuset.cpus: permission denied"
Oct 25 19:00:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:00:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:36.347721    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:00:36Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-db35cc655287784ee53cf7183b6a22602c1023901723750976084969b92af8e9.scope/cpuset.cpus: permission denied"
Oct 25 19:00:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:36.347790    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:00:36Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-db35cc655287784ee53cf7183b6a22602c1023901723750976084969b92af8e9.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:00:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:36.764032    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:37.761252    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:38.762668    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:39.766804    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:40.763829    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:41.150981    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:00:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:41.217372    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:41.217980    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:41.218080    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:41.218152    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:41.351768    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:00:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:00:41Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a9914918f5889b4353f7b31e0e2f677e0bb03dc55d46210f22c3fab1b0d7bdba.scope/cpuset.cpus: permission denied"
Oct 25 19:00:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:00:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:41.351935    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:00:41Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a9914918f5889b4353f7b31e0e2f677e0bb03dc55d46210f22c3fab1b0d7bdba.scope/cpuset.cpus: permission denied"
Oct 25 19:00:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:41.352009    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:00:41Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a9914918f5889b4353f7b31e0e2f677e0bb03dc55d46210f22c3fab1b0d7bdba.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:00:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:41.761241    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:00:42.161043    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:42.161087    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:42.534419    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:00:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:42.763508    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:42.764291    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:42.764321    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:42.764332    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:42.764352    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:42.765068    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:42.765175    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:43.096944    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:43.096975    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:43.762046    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:44.103125    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:00:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:44.762964    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:44.893129    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:44.893231    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:45.762013    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:46.761027    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:47.761336    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:00:48.346354    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:48.346416    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:48.761590    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:49.217463    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:49.217939    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:49.218039    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:49.218121    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:49.336670    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:00:49Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0efe4231f538091284c1c2c85623430eaf4453d58272a2c9e37836bfc6e5a94b.scope/cpuset.cpus: permission denied"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:49.336763    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:00:49Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0efe4231f538091284c1c2c85623430eaf4453d58272a2c9e37836bfc6e5a94b.scope/cpuset.cpus: permission denied"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:49.336808    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:00:49Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0efe4231f538091284c1c2c85623430eaf4453d58272a2c9e37836bfc6e5a94b.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:49.538583    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:49.761596    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:49.765689    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:49.766353    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:49.766380    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:49.766393    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:49.766444    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:49.770155    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:50.761993    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:51.161146    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:00:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:51.792279    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:52.200052    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:52.200099    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:00:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:52.763574    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:53.765186    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:54.103925    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:00:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:54.761796    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:55.762025    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:56.217681    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:56.218268    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:56.218358    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:56.218421    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:56.347715    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:00:56Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-acce810965adf3da4edd7e95b373874c6bfde0c397249e8498c5cc205cbcb064.scope/cpuset.cpus: permission denied"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:56.347812    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:00:56Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-acce810965adf3da4edd7e95b373874c6bfde0c397249e8498c5cc205cbcb064.scope/cpuset.cpus: permission denied"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:56.347879    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:00:56Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-acce810965adf3da4edd7e95b373874c6bfde0c397249e8498c5cc205cbcb064.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:56.542750    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:56.760856    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:56.771059    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:56.771973    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:56.772067    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:56.772140    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:56.772221    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:00:56.776070    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:00:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:57.762029    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:58.765048    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:00:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:00:59.761665    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:00.217427    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:00.217983    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:00.218082    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:00.218166    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:00.336505    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:01:00Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a9e0857e209b23e7a5f14e122047ff05516f259e5444d7b9a92dcf4ee162eebb.scope/cpuset.cpus: permission denied"
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:00.336633    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:01:00Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a9e0857e209b23e7a5f14e122047ff05516f259e5444d7b9a92dcf4ee162eebb.scope/cpuset.cpus: permission denied"
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:00.336697    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:01:00Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a9e0857e209b23e7a5f14e122047ff05516f259e5444d7b9a92dcf4ee162eebb.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:00.626552    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:00.626598    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:00.764548    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:01.166351    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:01:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:01.765745    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:02.762484    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:03.547438    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:01:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:03.674995    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:03.675121    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:03.761688    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:03.776809    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:03.777627    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:03.777659    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:03.777671    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:03.777690    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:03.782203    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:04.104721    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:01:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:04.761750    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:05.149539    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:05.149568    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:05.762098    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:01:06.650079    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:06.650233    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:06.763009    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:07.762252    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:08.761754    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:09.217933    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:09.218458    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:09.218561    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:09.218641    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:09.341551    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:01:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:01:09Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-754b898945a79f4b2cbe83ac0c476813ae7380cd86bd8619a2cd4a90650fed54.scope/cpuset.cpus: permission denied"
Oct 25 19:01:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:01:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:09.341644    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:01:09Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-754b898945a79f4b2cbe83ac0c476813ae7380cd86bd8619a2cd4a90650fed54.scope/cpuset.cpus: permission denied"
Oct 25 19:01:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:09.341698    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:01:09Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-754b898945a79f4b2cbe83ac0c476813ae7380cd86bd8619a2cd4a90650fed54.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:01:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:09.760466    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:10.553643    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:01:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:10.762096    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:10.783195    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:10.783822    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:10.783952    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:10.784043    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:10.784135    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:10.788432    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:11.171370    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:01:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:11.762489    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:12.761398    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:13.092098    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:13.092581    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:13.762683    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:14.105469    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:14.217797    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:14.218430    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:14.218467    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:14.218480    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:14.336348    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:01:14Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-40ee6a1a0339af83708f1f9eff8b506310a38678fc3523e0651c92a190bc2260.scope/cpuset.cpus: permission denied"
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:14.336449    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:01:14Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-40ee6a1a0339af83708f1f9eff8b506310a38678fc3523e0651c92a190bc2260.scope/cpuset.cpus: permission denied"
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:14.336497    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:01:14Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-40ee6a1a0339af83708f1f9eff8b506310a38678fc3523e0651c92a190bc2260.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:14.763277    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:14.896204    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:14.896253    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:15.761474    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:16.761407    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:17.558365    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:01:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:17.759752    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:17.788902    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:17.789523    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:17.789545    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:17.789557    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:17.789595    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:17.793508    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:01:17.975842    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:17.975885    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:18.762939    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:19.766115    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:20.762875    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:21.176390    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:01:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:21.761218    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:22.202796    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:22.202853    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:22.218220    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:22.218751    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:22.218774    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:22.218800    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:22.348693    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:01:22Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9db32aca6ad9272516ab705ce5ba7fc314a6f5f34aea8b4177f78ef77fe38d7f.scope/cpuset.cpus: permission denied"
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:22.348797    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:01:22Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9db32aca6ad9272516ab705ce5ba7fc314a6f5f34aea8b4177f78ef77fe38d7f.scope/cpuset.cpus: permission denied"
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:22.348869    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:01:22Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9db32aca6ad9272516ab705ce5ba7fc314a6f5f34aea8b4177f78ef77fe38d7f.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:01:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:22.762008    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:23.762684    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:23.795512    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:01:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:23.795734    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:01:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:24.106698    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:01:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:24.564002    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:01:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:24.765181    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:24.793791    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:24.794341    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:24.794436    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:24.794525    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:24.794638    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:24.799072    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:25.761899    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:26.760830    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:27.761001    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:28.761923    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:29.217870    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:29.218493    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:29.218593    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:29.218660    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:29.337315    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:01:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:01:29Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e205ea0d69d72e9497e115452a70f2027d6e68e768bb6219a82b8fdd0cf29650.scope/cpuset.cpus: permission denied"
Oct 25 19:01:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:01:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:29.337395    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:01:29Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e205ea0d69d72e9497e115452a70f2027d6e68e768bb6219a82b8fdd0cf29650.scope/cpuset.cpus: permission denied"
Oct 25 19:01:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:29.337438    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:01:29Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e205ea0d69d72e9497e115452a70f2027d6e68e768bb6219a82b8fdd0cf29650.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:01:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:29.765501    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:30.625627    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:30.625681    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:30.760763    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:31.181049    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:01:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:31.569852    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:01:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:31.761145    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:31.799295    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:31.799804    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:31.799918    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:31.799989    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:31.800115    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:31.804071    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:32.769654    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:33.670906    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:33.670940    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:33.761527    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:34.107506    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:01:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:34.217935    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:34.218557    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:34.218599    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:34.218611    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:34.340568    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:01:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:01:34Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-77c808325dca390b6a29a15507ce6118a2020b28b646f04347bb671226a0891f.scope/cpuset.cpus: permission denied"
Oct 25 19:01:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:01:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:34.340672    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:01:34Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-77c808325dca390b6a29a15507ce6118a2020b28b646f04347bb671226a0891f.scope/cpuset.cpus: permission denied"
Oct 25 19:01:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:34.340725    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:01:34Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-77c808325dca390b6a29a15507ce6118a2020b28b646f04347bb671226a0891f.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:01:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:34.764063    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:35.150038    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:35.150397    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:35.761668    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:36.761420    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:37.761306    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:38.574686    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:01:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:38.761438    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:38.804154    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:38.804788    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:38.804810    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:38.804851    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:38.804871    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:38.812584    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:39.760164    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:01:39.965364    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:39.965428    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:40.766176    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:41.189909    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:01:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:41.761957    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:42.762257    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:43.096846    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:43.097003    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:43.762285    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:44.108052    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:44.217772    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:44.218347    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:44.218456    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:44.218546    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:44.336493    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:01:44Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0babf33bc2c4c7106a773aedc16626823a2b9b85dbb333a91bff87cd0bc48131.scope/cpuset.cpus: permission denied"
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:44.336596    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:01:44Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0babf33bc2c4c7106a773aedc16626823a2b9b85dbb333a91bff87cd0bc48131.scope/cpuset.cpus: permission denied"
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:44.336650    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:01:44Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0babf33bc2c4c7106a773aedc16626823a2b9b85dbb333a91bff87cd0bc48131.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:01:44.482753    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:44.482802    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:44.760226    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:44.892802    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:44.892860    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:45.580860    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:01:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:45.765435    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:45.812755    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:45.813317    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:45.813408    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:45.813481    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:45.813553    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:45.813820    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:46.761741    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:01:47.384306    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:47.384442    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:47.762063    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:48.763648    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:49.762093    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:50.217558    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:50.218743    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:50.218765    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:50.218775    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:50.339717    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:01:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:01:50Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c1f94feb7c8930c1418893b1ecfc08429c9cf2c96fcef0615fb6b51ef276311b.scope/cpuset.cpus: permission denied"
Oct 25 19:01:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:01:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:50.340053    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:01:50Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c1f94feb7c8930c1418893b1ecfc08429c9cf2c96fcef0615fb6b51ef276311b.scope/cpuset.cpus: permission denied"
Oct 25 19:01:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:50.340109    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:01:50Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c1f94feb7c8930c1418893b1ecfc08429c9cf2c96fcef0615fb6b51ef276311b.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:01:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:50.762006    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:51.199296    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:01:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:51.199378    4021 event.go:228] Unable to write event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}' (retry limit exceeded!)
Oct 25 19:01:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:51.205442    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:01:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:51.762810    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:52.203379    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:52.203411    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:01:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:52.582693    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:01:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:52.760977    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:52.814184    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:52.814788    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:52.814923    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:52.815011    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:52.815098    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:52.818867    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:53.761483    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:54.108349    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:01:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:54.763169    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:55.761450    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:56.217327    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:56.218179    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:56.218215    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:56.218226    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:56.347601    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:01:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:01:56Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-b1dec42097cad214f7bace3be7a49ab162d27cc459d0ad649f356089f880dda7.scope/cpuset.cpus: permission denied"
Oct 25 19:01:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:01:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:56.347890    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:01:56Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-b1dec42097cad214f7bace3be7a49ab162d27cc459d0ad649f356089f880dda7.scope/cpuset.cpus: permission denied"
Oct 25 19:01:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:56.347938    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:01:56Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-b1dec42097cad214f7bace3be7a49ab162d27cc459d0ad649f356089f880dda7.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:01:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:56.763342    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:57.762173    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:58.274865    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:01:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:58.762404    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:59.587367    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:01:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:59.759368    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:01:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:59.819768    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:01:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:59.820435    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:01:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:59.820452    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:01:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:59.820462    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:01:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:01:59.820481    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:01:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:01:59.824505    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:00.629824    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:00.629876    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:00.764965    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:01.763768    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:02.762191    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:03.672845    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:03.673234    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:03.764329    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:02:03.764279    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:03.764521    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:04.108718    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:02:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:04.217580    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:04.218261    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:04.218360    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:04.218447    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:04.336592    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:02:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:02:04Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9555981e88df2b8a5992311177a9d9062060dba06b83bceed818163e132ce119.scope/cpuset.cpus: permission denied"
Oct 25 19:02:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:02:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:04.336691    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:02:04Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9555981e88df2b8a5992311177a9d9062060dba06b83bceed818163e132ce119.scope/cpuset.cpus: permission denied"
Oct 25 19:02:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:04.336739    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:02:04Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9555981e88df2b8a5992311177a9d9062060dba06b83bceed818163e132ce119.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:02:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:04.761983    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:05.149727    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:05.149888    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:05.762036    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:06.588925    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:02:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:06.760901    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:06.824798    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:06.825508    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:06.825535    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:06.825578    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:06.825600    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:06.829199    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:07.762438    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:08.280588    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:02:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:08.766213    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:09.762017    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:10.217475    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:10.218095    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:10.218117    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:10.218146    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:10.340555    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:02:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:02:10Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a57bd5179129926f5fc9c34a226c83162c6c0ba2b68d8c942073425f493bd35b.scope/cpuset.cpus: permission denied"
Oct 25 19:02:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:02:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:10.340662    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:02:10Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a57bd5179129926f5fc9c34a226c83162c6c0ba2b68d8c942073425f493bd35b.scope/cpuset.cpus: permission denied"
Oct 25 19:02:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:10.340716    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:02:10Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a57bd5179129926f5fc9c34a226c83162c6c0ba2b68d8c942073425f493bd35b.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:02:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:10.761475    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:11.761198    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:12.763153    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:13.096591    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:13.096643    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:13.594050    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:02:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:13.762392    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:13.829718    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:13.830287    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:13.830377    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:13.830453    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:13.830540    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:13.834175    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:14.109236    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:02:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:14.760759    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:14.891979    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:14.892028    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:15.218048    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:15.218526    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:15.218627    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:15.218721    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:15.341584    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:02:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:02:15Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-0336100f0c4c7b3741dfc67f2b1a98cef02a8d9f3c115bd1688f3a2254608385.scope/cpuset.cpus: permission denied"
Oct 25 19:02:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:02:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:15.341718    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:02:15Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-0336100f0c4c7b3741dfc67f2b1a98cef02a8d9f3c115bd1688f3a2254608385.scope/cpuset.cpus: permission denied"
Oct 25 19:02:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:15.341789    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:02:15Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-0336100f0c4c7b3741dfc67f2b1a98cef02a8d9f3c115bd1688f3a2254608385.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:02:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:15.765033    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:16.766579    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:17.761662    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:18.285067    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:02:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:18.761988    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:19.762208    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:20.599927    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:02:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:20.763081    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:20.834555    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:20.835124    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:20.835237    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:20.835342    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:20.835420    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:20.839688    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:21.217450    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:21.217845    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:21.217978    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:21.218060    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:21.340570    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:02:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:02:21Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-dc35e61248f25f1b274ad325d6f0204fbd13b21eec2833696743eca06ef88da7.scope/cpuset.cpus: permission denied"
Oct 25 19:02:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:02:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:21.340927    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:02:21Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-dc35e61248f25f1b274ad325d6f0204fbd13b21eec2833696743eca06ef88da7.scope/cpuset.cpus: permission denied"
Oct 25 19:02:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:21.340998    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:02:21Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-dc35e61248f25f1b274ad325d6f0204fbd13b21eec2833696743eca06ef88da7.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:02:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:21.766866    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:02:22.152539    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:22.152591    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:22.199066    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:22.199168    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:22.760982    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:23.761031    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:23.796346    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:02:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:23.796490    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:02:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:24.110164    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:02:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:24.762851    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:25.765721    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:26.761524    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:02:27.562404    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:27.562724    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:27.604246    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:02:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:27.762457    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:27.839845    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:27.840459    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:27.840553    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:27.840621    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:27.840692    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:27.848954    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:28.288684    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:02:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:28.764983    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:29.217394    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:29.218005    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:29.218025    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:29.218035    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:29.349620    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:02:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:02:29Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-2e1d7518d5d76872b8ed3731c57ac965d686115b219cedcd319633e8eb3b2c15.scope/cpuset.cpus: permission denied"
Oct 25 19:02:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:02:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:29.349732    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:02:29Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-2e1d7518d5d76872b8ed3731c57ac965d686115b219cedcd319633e8eb3b2c15.scope/cpuset.cpus: permission denied"
Oct 25 19:02:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:29.349784    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:02:29Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-2e1d7518d5d76872b8ed3731c57ac965d686115b219cedcd319633e8eb3b2c15.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:02:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:29.761570    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:30.627561    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:30.627714    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:30.762507    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:31.761275    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:32.761497    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:33.669718    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:33.669890    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:33.761535    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:34.110497    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:02:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:34.609113    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:02:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:34.760542    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:34.849904    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:34.850845    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:34.850933    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:34.851001    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:34.851074    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:34.854865    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:02:35.007993    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:35.008113    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:35.151114    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:35.151159    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:35.218129    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:35.218659    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:35.218742    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:35.218894    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:35.341542    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:02:35Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-f264f5c5fea5cc6b3aa9815ede8845404573d0c22bc712b0c57f396c968e8694.scope/cpuset.cpus: permission denied"
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:35.341641    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:02:35Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-f264f5c5fea5cc6b3aa9815ede8845404573d0c22bc712b0c57f396c968e8694.scope/cpuset.cpus: permission denied"
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:35.341690    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:02:35Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-f264f5c5fea5cc6b3aa9815ede8845404573d0c22bc712b0c57f396c968e8694.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:02:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:35.761123    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:36.757814    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:37.764893    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:38.293370    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:02:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:38.759138    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:39.763031    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:40.760682    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:41.615426    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:02:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:41.761927    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:41.855268    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:41.856062    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:41.856173    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:41.856270    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:41.856371    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:41.858019    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:42.760985    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:43.098954    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:43.099179    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:43.762125    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:44.111046    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:44.217862    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:44.218413    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:44.218438    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:44.218448    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:02:44.317434    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f57ef50bdfce72ac83f2bab1ba9382c37e0a8ea6b4e0bc3c6bcc47460f9e7336.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f57ef50bdfce72ac83f2bab1ba9382c37e0a8ea6b4e0bc3c6bcc47460f9e7336.scope: no such file or directory
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:44.345521    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:02:44Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f57ef50bdfce72ac83f2bab1ba9382c37e0a8ea6b4e0bc3c6bcc47460f9e7336.scope/cpuset.cpus: permission denied"
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:44.345625    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:02:44Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f57ef50bdfce72ac83f2bab1ba9382c37e0a8ea6b4e0bc3c6bcc47460f9e7336.scope/cpuset.cpus: permission denied"
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:44.345679    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:02:44Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f57ef50bdfce72ac83f2bab1ba9382c37e0a8ea6b4e0bc3c6bcc47460f9e7336.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:44.761171    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:44.893271    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:44.893688    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:45.761353    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:46.217779    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:46.218435    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:46.218544    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:46.218613    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:46.336415    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:02:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:02:46Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-05a57aed1e70c37e97bf0eb893ffc07fdc89458c7aaf2c146671163db56d7b28.scope/cpuset.cpus: permission denied"
Oct 25 19:02:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:02:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:46.336730    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:02:46Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-05a57aed1e70c37e97bf0eb893ffc07fdc89458c7aaf2c146671163db56d7b28.scope/cpuset.cpus: permission denied"
Oct 25 19:02:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:46.336786    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:02:46Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-05a57aed1e70c37e97bf0eb893ffc07fdc89458c7aaf2c146671163db56d7b28.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:02:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:46.763297    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:47.762269    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:48.298026    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:02:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:48.620658    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:02:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:48.762407    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:48.858543    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:48.859300    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:48.859396    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:48.859470    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:48.859550    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:48.864092    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:49.761617    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:50.761991    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:51.762383    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:52.202989    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:52.203114    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:02:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:52.762414    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:53.760772    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:54.111939    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:02:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:54.761534    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:55.629196    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:02:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:55.761273    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:55.864131    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:55.864788    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:55.864938    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:55.865040    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:55.865122    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:55.869145    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:02:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:56.217402    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:56.217818    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:56.217932    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:56.218006    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:56.336505    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:02:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:02:56Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a79e7dee25e4fba5e5d5dea986ad3046334400bdf5a9f8a695984be1651e6cfb.scope/cpuset.cpus: permission denied"
Oct 25 19:02:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:02:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:56.336599    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:02:56Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a79e7dee25e4fba5e5d5dea986ad3046334400bdf5a9f8a695984be1651e6cfb.scope/cpuset.cpus: permission denied"
Oct 25 19:02:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:56.336651    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:02:56Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a79e7dee25e4fba5e5d5dea986ad3046334400bdf5a9f8a695984be1651e6cfb.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:02:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:56.761358    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:57.763871    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:58.218074    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:02:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:58.218835    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:02:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:58.218856    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:02:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:58.218866    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:02:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:58.306828    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:02:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:58.341432    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:02:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:02:58Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-95b19f31ae2c5098d2b10f007377d4a8b8455c7e3892442baefd36f92debacaf.scope/cpuset.cpus: permission denied"
Oct 25 19:02:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:02:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:58.341527    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:02:58Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-95b19f31ae2c5098d2b10f007377d4a8b8455c7e3892442baefd36f92debacaf.scope/cpuset.cpus: permission denied"
Oct 25 19:02:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:02:58.341580    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:02:58Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-95b19f31ae2c5098d2b10f007377d4a8b8455c7e3892442baefd36f92debacaf.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:02:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:58.761226    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:02:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:02:59.761621    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:00.627530    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:00.627563    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:00.760934    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:01.761472    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:02.634638    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:03:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:02.761389    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:03:02.778394    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:02.778766    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:02.869693    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:02.870329    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:02.870430    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:02.870504    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:02.870584    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:02.874965    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:03.671932    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:03.672068    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:03.762099    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:04.113291    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:03:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:04.763337    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:05.152000    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:05.152318    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:05.762421    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:06.761962    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:07.764065    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:08.217543    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:08.218188    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:08.218209    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:08.218221    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:08.309066    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:03:08.313832    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-85af243e1f2da02e1f0b9ba112b7d3b2e084e43f5a46910ac583ea0090ef2333.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-85af243e1f2da02e1f0b9ba112b7d3b2e084e43f5a46910ac583ea0090ef2333.scope: no such file or directory
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:08.347606    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:03:08Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-85af243e1f2da02e1f0b9ba112b7d3b2e084e43f5a46910ac583ea0090ef2333.scope/cpuset.cpus: permission denied"
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:08.347714    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:03:08Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-85af243e1f2da02e1f0b9ba112b7d3b2e084e43f5a46910ac583ea0090ef2333.scope/cpuset.cpus: permission denied"
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:08.347768    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:03:08Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-85af243e1f2da02e1f0b9ba112b7d3b2e084e43f5a46910ac583ea0090ef2333.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:03:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:08.761854    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:09.641391    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:03:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:09.765195    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:09.875569    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:09.876203    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:09.876337    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:09.876411    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:09.876490    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:09.882950    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:10.765104    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:11.761703    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:12.761943    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:13.096544    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:13.096576    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:03:13.194254    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:13.194298    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:13.762527    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:14.114166    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:14.217367    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:14.217936    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:14.218041    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:14.218132    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:14.336378    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:03:14Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-82edc8dee2fe450f6386793ce004797793aecb386fe633f5a7ba4bc52cd887a2.scope/cpuset.cpus: permission denied"
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:14.336487    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:03:14Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-82edc8dee2fe450f6386793ce004797793aecb386fe633f5a7ba4bc52cd887a2.scope/cpuset.cpus: permission denied"
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:14.336539    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:03:14Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-82edc8dee2fe450f6386793ce004797793aecb386fe633f5a7ba4bc52cd887a2.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:14.764627    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:14.892574    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:14.892607    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:15.762301    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:16.645840    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:03:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:16.761281    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:16.883173    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:16.883730    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:16.883763    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:16.883771    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:16.883790    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:16.887970    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:17.761281    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:18.313640    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:03:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:18.760708    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:19.761393    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:20.762632    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:21.761695    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:22.200936    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:22.200988    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:22.760928    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.217457    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.218175    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.218194    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.218204    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:23.345497    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:03:23Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-97e1358576508685b79273bee723250e850ecbb399ac4a238c4a38a0b44baded.scope/cpuset.cpus: permission denied"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:23.345592    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:03:23Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-97e1358576508685b79273bee723250e850ecbb399ac4a238c4a38a0b44baded.scope/cpuset.cpus: permission denied"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:23.345637    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:03:23Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-97e1358576508685b79273bee723250e850ecbb399ac4a238c4a38a0b44baded.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:23.652455    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.758303    4021 kubelet.go:1454] "Image garbage collection succeeded"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.762474    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.796691    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.796721    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.888411    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.889004    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.889035    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.889046    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:23.889066    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:23.894282    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:24.114619    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:03:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:03:24.543900    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:24.544209    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:24.765735    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:25.762271    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:26.217648    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:26.218184    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:26.218287    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:26.218417    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:26.336405    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:03:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:03:26Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-db10cdf26cebe80f998db91b94f9452ecbce25a022e225670a34b9eb41621e65.scope/cpuset.cpus: permission denied"
Oct 25 19:03:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:03:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:26.336544    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:03:26Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-db10cdf26cebe80f998db91b94f9452ecbce25a022e225670a34b9eb41621e65.scope/cpuset.cpus: permission denied"
Oct 25 19:03:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:26.336598    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:03:26Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-db10cdf26cebe80f998db91b94f9452ecbce25a022e225670a34b9eb41621e65.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:03:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:26.761885    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:03:27.469134    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:27.469486    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:27.762054    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:28.318220    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:03:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:28.761726    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:29.762221    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:30.629403    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:30.629453    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:30.659750    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:03:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:30.764633    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:30.894923    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:30.895520    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:30.895652    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:30.895729    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:30.895808    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:30.901996    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:31.762758    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:32.760793    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:33.670083    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:33.670132    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:33.762398    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:34.114978    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:03:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:34.763956    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:35.150467    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:35.150495    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:35.762079    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:37.346249    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:37.665622    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:03:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:37.761718    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:37.903110    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:37.903717    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:37.903736    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:37.903748    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:37.903783    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:37.907540    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:38.218077    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:38.218502    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:38.218517    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:38.218528    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:03:38.312693    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-59ff49e3ac9a62429e7fb6e60d68ce8ef128a5d72e26276641bf2feb9fba1304.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-59ff49e3ac9a62429e7fb6e60d68ce8ef128a5d72e26276641bf2feb9fba1304.scope: no such file or directory
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:38.324864    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:38.324948    4021 event.go:228] Unable to write event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}' (retry limit exceeded!)
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:38.328207    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:38.341497    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:03:38Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-59ff49e3ac9a62429e7fb6e60d68ce8ef128a5d72e26276641bf2feb9fba1304.scope/cpuset.cpus: permission denied"
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:38.341619    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:03:38Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-59ff49e3ac9a62429e7fb6e60d68ce8ef128a5d72e26276641bf2feb9fba1304.scope/cpuset.cpus: permission denied"
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:38.341662    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:03:38Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-59ff49e3ac9a62429e7fb6e60d68ce8ef128a5d72e26276641bf2feb9fba1304.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:03:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:38.762700    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:39.483079    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:03:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:39.762760    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:40.218229    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:40.218848    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:40.218873    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:40.218910    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:40.341440    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:03:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:03:40Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-2f3d834d5eedf14c56e5f761fce9eff989b497ca2dc2682adcbcb673d1df182b.scope/cpuset.cpus: permission denied"
Oct 25 19:03:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:03:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:40.341543    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:03:40Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-2f3d834d5eedf14c56e5f761fce9eff989b497ca2dc2682adcbcb673d1df182b.scope/cpuset.cpus: permission denied"
Oct 25 19:03:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:40.341596    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:03:40Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-2f3d834d5eedf14c56e5f761fce9eff989b497ca2dc2682adcbcb673d1df182b.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:03:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:40.761714    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:41.761911    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:42.762144    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:43.096745    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:43.096903    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:43.761919    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:44.115228    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:03:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:44.670039    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:03:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:44.762195    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:44.896023    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:44.896054    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:44.908072    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:44.908716    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:44.908737    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:44.908748    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:44.908767    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:44.912581    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:45.765703    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:03:45.774650    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:45.774787    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:46.776058    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:47.763131    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:48.762001    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:49.489972    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:03:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:49.763279    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:50.761744    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:51.677672    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:03:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:51.763059    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:51.913037    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:51.913636    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:51.913742    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:51.913835    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:51.913948    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:51.918585    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:52.201984    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:52.202178    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:52.217740    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:52.218937    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:52.223985    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:52.224029    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:52.224040    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:52.224351    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:52.224435    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:52.224506    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:52.423126    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:03:52Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-296063474db5175de718b6471baaa74e07d336236e2de875ccc6e42d5695fbba.scope/cpuset.cpus: permission denied"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:52.423438    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:03:52Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-296063474db5175de718b6471baaa74e07d336236e2de875ccc6e42d5695fbba.scope/cpuset.cpus: permission denied"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:52.423499    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:03:52Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-296063474db5175de718b6471baaa74e07d336236e2de875ccc6e42d5695fbba.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:52.438322    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:03:52Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-4c499976e1066659bfee7fa228cec53bb9bc855d687a65e66ae54b08d61fb3c3.scope/cpuset.cpus: permission denied"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:52.438405    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:03:52Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-4c499976e1066659bfee7fa228cec53bb9bc855d687a65e66ae54b08d61fb3c3.scope/cpuset.cpus: permission denied"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:52.438462    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:03:52Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-4c499976e1066659bfee7fa228cec53bb9bc855d687a65e66ae54b08d61fb3c3.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:03:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:52.761410    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:03:53.103836    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:53.103985    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:53.761535    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:54.115755    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:03:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:54.761145    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:55.762036    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:56.761020    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:57.765147    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:58.682507    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:03:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:58.761041    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:03:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:58.919569    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:03:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:58.920467    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:03:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:58.920496    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:03:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:58.920528    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:03:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:58.920550    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:58.924227    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:03:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:03:59.494523    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:03:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:03:59.761873    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:00.630748    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:00.631248    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:00.768631    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:01.762415    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:02.761867    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:03.672594    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:03.673043    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:03.765268    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:04.116205    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:04:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:04.218182    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:04.218653    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:04.218747    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:04.218839    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:04.341448    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:04:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:04:04Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-9db28bf177e95824a1719667bb07742ca7b64890db757d617c3e5a4b764a089f.scope/cpuset.cpus: permission denied"
Oct 25 19:04:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:04:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:04.341549    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:04:04Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-9db28bf177e95824a1719667bb07742ca7b64890db757d617c3e5a4b764a089f.scope/cpuset.cpus: permission denied"
Oct 25 19:04:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:04.341598    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:04:04Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-9db28bf177e95824a1719667bb07742ca7b64890db757d617c3e5a4b764a089f.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:04:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:04.762814    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:05.151377    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:05.151428    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:05.217848    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:05.218467    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:05.218500    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:05.218513    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:05.342794    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:04:05Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-899a87425493f04a11570dc5957eb018da33d1ce6f40b216264bf8cdc5ea6d8d.scope/cpuset.cpus: permission denied"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:05.342922    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:04:05Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-899a87425493f04a11570dc5957eb018da33d1ce6f40b216264bf8cdc5ea6d8d.scope/cpuset.cpus: permission denied"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:05.342989    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:04:05Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-899a87425493f04a11570dc5957eb018da33d1ce6f40b216264bf8cdc5ea6d8d.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:05.692619    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:05.760851    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:05.924452    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:05.925011    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:05.925030    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:05.925039    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:05.925073    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:05.930269    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:06.762017    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:07.763422    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:04:08.419913    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:08.419963    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:08.761293    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:09.499158    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:04:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:09.761481    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:10.761135    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:11.766293    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:12.700590    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:04:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:12.766813    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:12.931219    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:12.931748    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:12.931842    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:12.931948    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:12.932031    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:12.937090    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:13.093981    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:13.094049    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:13.766276    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:14.117153    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:04:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:14.764959    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:14.892235    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:14.892335    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:15.761631    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:16.217545    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:16.218126    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:16.218215    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:16.218282    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:16.339569    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:04:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:04:16Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-6c8a86038180adde411ca5c7e669e8f29b6302013937f026bd98e0dd76d338b7.scope/cpuset.cpus: permission denied"
Oct 25 19:04:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:04:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:16.339677    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:04:16Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-6c8a86038180adde411ca5c7e669e8f29b6302013937f026bd98e0dd76d338b7.scope/cpuset.cpus: permission denied"
Oct 25 19:04:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:16.339732    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:04:16Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-6c8a86038180adde411ca5c7e669e8f29b6302013937f026bd98e0dd76d338b7.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:04:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:16.764434    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:17.761359    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:18.763505    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:19.504920    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:04:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:19.715275    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:04:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:19.764795    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:19.937845    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:19.938406    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:19.938425    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:19.938436    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:19.938454    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:19.943162    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:20.218071    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:20.218502    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:20.218527    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:20.218538    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:04:20.330644    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:20.330695    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:20.344978    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:04:20Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e5e28ad43f8ec7e445c849866db238d1d7566d78fedc2aff49ba8c0218ca2aad.scope/cpuset.cpus: permission denied"
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:20.345075    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:04:20Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e5e28ad43f8ec7e445c849866db238d1d7566d78fedc2aff49ba8c0218ca2aad.scope/cpuset.cpus: permission denied"
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:20.345130    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:04:20Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e5e28ad43f8ec7e445c849866db238d1d7566d78fedc2aff49ba8c0218ca2aad.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:04:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:20.764922    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:21.762515    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:22.198221    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:22.198252    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:22.761323    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:23.774600    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:23.796813    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:04:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:23.796931    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:04:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:24.117797    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:04:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:24.761628    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:25.764772    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:26.720618    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:04:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:26.764045    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:26.943998    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:26.944726    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:26.944902    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:26.944987    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:26.945075    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:26.949118    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:27.765041    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:28.797396    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:29.532855    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:04:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:29.843208    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:30.217770    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:30.218315    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:30.218410    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:30.218485    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:30.339617    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:04:30Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-dc89f396a6cb68bd0f53f2884c02d233d9dd47cad4c42f42bc57bbdccb633aa8.scope/cpuset.cpus: permission denied"
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:30.339724    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:04:30Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-dc89f396a6cb68bd0f53f2884c02d233d9dd47cad4c42f42bc57bbdccb633aa8.scope/cpuset.cpus: permission denied"
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:30.339777    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:04:30Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-dc89f396a6cb68bd0f53f2884c02d233d9dd47cad4c42f42bc57bbdccb633aa8.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:30.704354    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:30.704387    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:30.768123    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:31.218005    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:31.218854    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:31.218956    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:31.219030    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:04:31.310587    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9b8a8fea8d4e804130aee1f5d1fb3bdf5441665902a07a192ee7ba1ba52d6d61.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9b8a8fea8d4e804130aee1f5d1fb3bdf5441665902a07a192ee7ba1ba52d6d61.scope: no such file or directory
Oct 25 19:04:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:31.344548    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:04:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:04:31Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9b8a8fea8d4e804130aee1f5d1fb3bdf5441665902a07a192ee7ba1ba52d6d61.scope/cpuset.cpus: permission denied"
Oct 25 19:04:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:04:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:31.344662    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:04:31Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9b8a8fea8d4e804130aee1f5d1fb3bdf5441665902a07a192ee7ba1ba52d6d61.scope/cpuset.cpus: permission denied"
Oct 25 19:04:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:31.344714    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:04:31Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9b8a8fea8d4e804130aee1f5d1fb3bdf5441665902a07a192ee7ba1ba52d6d61.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:04:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:31.764030    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:32.763203    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:33.673716    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:33.673875    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:33.726758    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:04:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:33.763771    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:33.949639    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:33.950245    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:33.950268    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:33.950278    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:33.950310    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:33.955283    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:34.118706    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:04:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:34.761561    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:35.154906    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:35.154956    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:35.765865    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:37.477679    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:37.940045    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:38.762619    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:39.537765    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:04:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:39.761972    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:04:40.206658    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:40.207007    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:40.956136    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:40.956780    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:40.956803    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:40.956815    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:40.956856    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:41.653416    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:41.653506    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:04:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:41.653557    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:41.759253    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:42.218214    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:42.218653    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:42.218675    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:42.218684    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:42.337458    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:04:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:04:42Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a6e5de973d6715e9fd26bf900298f0072a38a015a0d3840e189d68c8d2377d06.scope/cpuset.cpus: permission denied"
Oct 25 19:04:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:04:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:42.337570    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:04:42Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a6e5de973d6715e9fd26bf900298f0072a38a015a0d3840e189d68c8d2377d06.scope/cpuset.cpus: permission denied"
Oct 25 19:04:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:42.337617    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:04:42Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a6e5de973d6715e9fd26bf900298f0072a38a015a0d3840e189d68c8d2377d06.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:04:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:43.782524    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:43.782899    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:43.783110    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:44.119311    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:44.218092    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:44.218636    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:44.218728    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:44.218801    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:44.336316    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:04:44Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d57bb40c1de1c63b207b1ce635a1673396602cde16bfd54bce6f9b0ea22ef841.scope/cpuset.cpus: permission denied"
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:44.336444    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:04:44Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d57bb40c1de1c63b207b1ce635a1673396602cde16bfd54bce6f9b0ea22ef841.scope/cpuset.cpus: permission denied"
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:44.336508    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:04:44Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d57bb40c1de1c63b207b1ce635a1673396602cde16bfd54bce6f9b0ea22ef841.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:04:44.610227    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:44.610278    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:44.767803    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:44.891738    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:44.892085    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:45.763149    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:46.762048    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:47.763513    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:48.653456    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:48.654333    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:48.654353    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:48.654364    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:48.654402    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:48.658557    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:04:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:48.658594    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:48.762730    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:49.542612    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:04:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:49.763330    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:50.761859    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:51.765294    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:52.200913    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:52.200949    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:04:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:52.761482    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:53.762788    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:54.120060    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:04:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:54.762335    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.217812    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.217812    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.218788    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.218904    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.218996    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.218950    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.219157    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.219170    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:55.422904    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:04:55Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-ccb248468d0558939d86e42b773489886bcc64212328115b2ece7f387d0eeacb.scope/cpuset.cpus: permission denied"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:55.423073    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:04:55Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-ccb248468d0558939d86e42b773489886bcc64212328115b2ece7f387d0eeacb.scope/cpuset.cpus: permission denied"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:55.423154    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:04:55Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-ccb248468d0558939d86e42b773489886bcc64212328115b2ece7f387d0eeacb.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:55.438636    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:04:55Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-40e9600e837ee058d27c828a299d6aeaf3d882483d20142e1558ebbf89dff9bb.scope/cpuset.cpus: permission denied"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:55.438713    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:04:55Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-40e9600e837ee058d27c828a299d6aeaf3d882483d20142e1558ebbf89dff9bb.scope/cpuset.cpus: permission denied"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:55.438754    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:04:55Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-40e9600e837ee058d27c828a299d6aeaf3d882483d20142e1558ebbf89dff9bb.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.659171    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.660108    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.660132    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.660141    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.660158    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:55.666034    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:55.666070    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:04:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:55.762269    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:56.762878    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:04:57.493226    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:57.493290    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:57.762240    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:58.762527    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:04:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:04:59.547260    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:04:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:04:59.763928    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:00.630212    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:00.630261    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:00.762658    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:01.762756    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:05:02.463476    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:02.463538    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:02.666497    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:02.667381    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:02.667425    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:02.667440    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:02.667462    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:02.671085    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:02.671206    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:05:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:02.763124    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:03.670540    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:03.670676    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:03.763234    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:04.120165    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:05:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:04.762042    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:05.150715    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:05.151073    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:05.764977    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:06.218053    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:06.218647    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:06.218747    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:06.218853    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:06.341534    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:05:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:05:06Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-3526dd7fce785f6ed262278f4b4a17b211d04c9fe9a7d61b672c64b48c340e9d.scope/cpuset.cpus: permission denied"
Oct 25 19:05:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:05:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:06.341617    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:05:06Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-3526dd7fce785f6ed262278f4b4a17b211d04c9fe9a7d61b672c64b48c340e9d.scope/cpuset.cpus: permission denied"
Oct 25 19:05:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:06.341662    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:05:06Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-3526dd7fce785f6ed262278f4b4a17b211d04c9fe9a7d61b672c64b48c340e9d.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:05:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:06.760677    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:07.217560    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:07.218204    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:07.218651    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:07.218739    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:07.343709    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:05:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:05:07Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-af7db656089dd9bdb3cd4e3e24758ff07531ef259593016c5cd016abbe81be9a.scope/cpuset.cpus: permission denied"
Oct 25 19:05:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:05:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:07.343810    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:05:07Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-af7db656089dd9bdb3cd4e3e24758ff07531ef259593016c5cd016abbe81be9a.scope/cpuset.cpus: permission denied"
Oct 25 19:05:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:07.343863    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:05:07Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-af7db656089dd9bdb3cd4e3e24758ff07531ef259593016c5cd016abbe81be9a.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:05:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:07.761260    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:08.763325    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:09.551729    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:05:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:09.671695    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:09.672359    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:09.672501    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:09.672572    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:09.672648    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:09.679859    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:09.679908    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:05:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:09.761031    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:10.761276    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:11.764766    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:12.763261    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:13.098241    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:13.098570    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:13.757643    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:14.121279    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:05:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:14.761930    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:14.893648    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:14.893765    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:15.765560    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:16.680753    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:16.681574    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:16.681667    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:16.681744    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:16.681831    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:16.685015    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:16.685134    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:05:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:16.761877    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:17.765366    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:18.217226    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:18.217728    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:18.217841    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:18.217934    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:18.337879    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:05:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:05:18Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-baeb1fcb9156f6cc55091c03a918d7f147466439ee269f8efec75ff3442a8df0.scope/cpuset.cpus: permission denied"
Oct 25 19:05:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:05:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:18.338067    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:05:18Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-baeb1fcb9156f6cc55091c03a918d7f147466439ee269f8efec75ff3442a8df0.scope/cpuset.cpus: permission denied"
Oct 25 19:05:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:18.338139    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:05:18Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-baeb1fcb9156f6cc55091c03a918d7f147466439ee269f8efec75ff3442a8df0.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:05:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:18.761411    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:19.217834    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:19.218584    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:19.218664    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:19.218742    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:19.340595    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:05:19Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-7ec32aaa1a970c4c6d63017cbaec30575e571438985046ed9eebae8ca4f54b33.scope/cpuset.cpus: permission denied"
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:19.340731    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:05:19Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-7ec32aaa1a970c4c6d63017cbaec30575e571438985046ed9eebae8ca4f54b33.scope/cpuset.cpus: permission denied"
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:19.340802    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:05:19Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-7ec32aaa1a970c4c6d63017cbaec30575e571438985046ed9eebae8ca4f54b33.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:19.556969    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:19.557056    4021 event.go:228] Unable to write event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}' (retry limit exceeded!)
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:19.565734    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:05:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:19.761592    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:20.183467    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:05:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:20.761505    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:21.760429    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:22.199037    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:22.199072    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:22.761189    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:23.685603    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:23.686456    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:23.686474    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:23.686485    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:23.686521    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:23.693299    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:23.693373    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:05:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:23.762923    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:23.797071    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:05:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:23.797119    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:05:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:24.121633    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:05:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:24.761433    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:25.764353    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:26.760902    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:27.763345    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:28.765876    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:29.761859    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:30.187810    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:30.218159    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:30.218839    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:30.218862    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:30.218891    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:05:30.310501    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d367a3b1982ef5c4262b53ea15e208d2dad76013fb8a0ccd551e1adcc6cb6e56.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d367a3b1982ef5c4262b53ea15e208d2dad76013fb8a0ccd551e1adcc6cb6e56.scope: no such file or directory
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:30.344565    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:05:30Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d367a3b1982ef5c4262b53ea15e208d2dad76013fb8a0ccd551e1adcc6cb6e56.scope/cpuset.cpus: permission denied"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:30.344679    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:05:30Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d367a3b1982ef5c4262b53ea15e208d2dad76013fb8a0ccd551e1adcc6cb6e56.scope/cpuset.cpus: permission denied"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:30.344726    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:05:30Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-d367a3b1982ef5c4262b53ea15e208d2dad76013fb8a0ccd551e1adcc6cb6e56.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:30.626755    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:30.626806    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:30.693846    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:30.699488    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:30.699512    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:30.699522    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:30.699541    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:30.700069    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:30.700183    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:05:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:30.758146    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:31.762896    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:32.758805    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:33.218008    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:33.218558    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:33.218579    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:33.218590    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:33.337446    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:05:33Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e31f799266a5125b22a8dc4818da78d00f6912d296a182252137c31ca6ddc45f.scope/cpuset.cpus: permission denied"
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:33.337548    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:05:33Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e31f799266a5125b22a8dc4818da78d00f6912d296a182252137c31ca6ddc45f.scope/cpuset.cpus: permission denied"
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:33.337599    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:05:33Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-e31f799266a5125b22a8dc4818da78d00f6912d296a182252137c31ca6ddc45f.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:33.670637    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:33.670667    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:33.762196    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:34.122454    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:05:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:34.761911    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:35.150013    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:35.150065    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:35.761105    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:36.761558    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:37.700910    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:37.701581    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:37.701713    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:37.701819    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:37.701926    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:37.705386    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:37.705400    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:05:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:37.761709    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:05:38.588093    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:38.588157    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:38.763107    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:39.766911    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:40.192860    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:05:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:40.762109    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:41.765647    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:05:42.140848    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:42.141010    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:42.762712    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:43.096595    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:43.096927    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:43.217640    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:43.218222    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:43.218242    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:43.218253    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:43.336877    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:05:43Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9d16150db0295be21b8ead068a2a38b84062867f6220507e4202cdadde49d941.scope/cpuset.cpus: permission denied"
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:43.336991    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:05:43Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9d16150db0295be21b8ead068a2a38b84062867f6220507e4202cdadde49d941.scope/cpuset.cpus: permission denied"
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:43.337043    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:05:43Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-9d16150db0295be21b8ead068a2a38b84062867f6220507e4202cdadde49d941.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:05:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:43.761016    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:44.123436    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:05:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:44.706330    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:44.707076    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:44.707101    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:44.707111    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:44.707129    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:44.710191    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:44.710328    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:05:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:44.762815    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:44.901174    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:44.901290    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:45.761369    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:46.217618    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:46.218164    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:46.218182    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:46.218193    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:05:46.315324    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-9b9a3b9221a80e1730114c536f1d63b327a42fda6ce3e40ab96d2f29bb781b5a.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-9b9a3b9221a80e1730114c536f1d63b327a42fda6ce3e40ab96d2f29bb781b5a.scope: no such file or directory
Oct 25 19:05:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:46.344459    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:05:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:05:46Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-9b9a3b9221a80e1730114c536f1d63b327a42fda6ce3e40ab96d2f29bb781b5a.scope/cpuset.cpus: permission denied"
Oct 25 19:05:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:05:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:46.344561    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:05:46Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-9b9a3b9221a80e1730114c536f1d63b327a42fda6ce3e40ab96d2f29bb781b5a.scope/cpuset.cpus: permission denied"
Oct 25 19:05:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:46.344616    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:05:46Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-9b9a3b9221a80e1730114c536f1d63b327a42fda6ce3e40ab96d2f29bb781b5a.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:05:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:46.760625    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:47.759693    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:48.761408    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:05:48.963926    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:48.963970    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:49.766330    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:50.198274    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:05:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:50.761848    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:51.710414    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:51.711257    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:51.711284    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:51.711296    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:51.711347    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:51.711940    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:51.712040    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:05:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:51.761522    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:05:51.869921    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:51.869980    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:52.203438    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:52.203544    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:05:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:52.761376    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:53.761184    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:54.123679    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:05:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:54.762159    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:55.761337    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:56.762601    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:57.761912    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:58.217863    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:58.218412    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:58.218500    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:58.218571    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:58.341455    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:05:58Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-681d6390c25ebed6d1428618ff90963e66dffe10fae523a5960cfed70394a82f.scope/cpuset.cpus: permission denied"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:58.341595    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:05:58Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-681d6390c25ebed6d1428618ff90963e66dffe10fae523a5960cfed70394a82f.scope/cpuset.cpus: permission denied"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:58.341641    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:05:58Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-681d6390c25ebed6d1428618ff90963e66dffe10fae523a5960cfed70394a82f.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:58.712102    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:58.713065    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:58.713196    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:58.713283    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:58.713380    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:58.720239    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:05:58.720362    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:05:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:58.761398    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:05:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:05:59.761254    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:00.203973    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:06:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:00.630423    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:00.630476    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:00.764665    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:01.217602    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:01.218194    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:01.218283    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:01.218354    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:01.344027    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:06:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:06:01Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-54ac36b5988976bdc562828ee39008b7ecb188b273b09f3b0a56d1582a29beff.scope/cpuset.cpus: permission denied"
Oct 25 19:06:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:06:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:01.344121    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:06:01Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-54ac36b5988976bdc562828ee39008b7ecb188b273b09f3b0a56d1582a29beff.scope/cpuset.cpus: permission denied"
Oct 25 19:06:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:01.344174    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:06:01Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-54ac36b5988976bdc562828ee39008b7ecb188b273b09f3b0a56d1582a29beff.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:06:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:01.760360    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:02.763666    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:03.669713    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:03.669758    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:03.762849    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:04.124231    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:06:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:04.761718    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:05.150530    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:05.150839    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:05.721360    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:05.722100    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:05.722125    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:05.722137    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:05.722175    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:05.725759    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:05.725790    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:06:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:05.762463    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:06.764811    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:07.761927    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:08.760882    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:09.765246    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:10.209436    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:06:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:10.761416    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:11.764335    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:12.217867    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:12.218389    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:12.218531    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:12.218609    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:12.352434    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:06:12Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c556d591306dd13a48f7906fa49e68326d61fdbd783edc768fdf4f3e2a6281b2.scope/cpuset.cpus: permission denied"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:12.352530    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:06:12Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c556d591306dd13a48f7906fa49e68326d61fdbd783edc768fdf4f3e2a6281b2.scope/cpuset.cpus: permission denied"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:12.352577    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:06:12Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c556d591306dd13a48f7906fa49e68326d61fdbd783edc768fdf4f3e2a6281b2.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:12.726772    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:12.727479    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:12.727499    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:12.727510    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:12.727530    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:12.734012    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:12.734076    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:06:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:12.761283    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:13.096253    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:13.096282    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:13.762858    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:14.124920    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:14.217848    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:14.218297    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:14.218402    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:14.218477    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:14.341632    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:06:14Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-2c7fdb5769b261541a4e5c94880d813cbb932a2cdbe5f32b66d9b1699980b37e.scope/cpuset.cpus: permission denied"
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:14.341741    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:06:14Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-2c7fdb5769b261541a4e5c94880d813cbb932a2cdbe5f32b66d9b1699980b37e.scope/cpuset.cpus: permission denied"
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:14.341796    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:06:14Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-2c7fdb5769b261541a4e5c94880d813cbb932a2cdbe5f32b66d9b1699980b37e.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:14.763142    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:14.892316    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:14.892363    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:15.762313    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:16.763462    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:17.760544    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:18.761420    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:19.735041    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:19.735647    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:19.735673    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:19.735684    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:19.735702    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:19.740139    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:19.740250    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:06:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:19.762152    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:20.214189    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:06:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:20.761627    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:21.762578    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:22.199512    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:22.199673    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:22.764552    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:23.760966    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:23.797624    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:06:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:23.797942    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:06:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:24.125655    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:06:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:24.762980    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:25.760943    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:26.217253    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:26.218049    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:26.218093    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:26.218106    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:26.350396    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:06:26Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0b3e5bc42d0d1396ae1ec8e40824e05c2f000667b8184d6ec26b0d66c5c4729f.scope/cpuset.cpus: permission denied"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:26.350509    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:06:26Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0b3e5bc42d0d1396ae1ec8e40824e05c2f000667b8184d6ec26b0d66c5c4729f.scope/cpuset.cpus: permission denied"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:26.350564    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:06:26Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-0b3e5bc42d0d1396ae1ec8e40824e05c2f000667b8184d6ec26b0d66c5c4729f.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:26.741048    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:26.741732    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:26.741751    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:26.741762    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:26.741781    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:26.747753    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:26.747806    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:06:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:26.761543    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:27.218213    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:27.218685    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:27.218772    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:27.218859    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:27.340840    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:06:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:06:27Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8110d7dc965b244a92bd5b7ff8d001b5978fcfaa37129763cede1e97503f1423.scope/cpuset.cpus: permission denied"
Oct 25 19:06:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:06:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:27.340948    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:06:27Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8110d7dc965b244a92bd5b7ff8d001b5978fcfaa37129763cede1e97503f1423.scope/cpuset.cpus: permission denied"
Oct 25 19:06:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:27.341015    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:06:27Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8110d7dc965b244a92bd5b7ff8d001b5978fcfaa37129763cede1e97503f1423.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:06:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:27.762958    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:28.761197    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:06:29.674358    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:29.674695    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:29.761893    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:30.220186    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:06:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:30.639856    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:30.639903    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:30.761618    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:06:30.853209    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:30.853254    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:31.762127    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:32.763044    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:33.669279    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:33.669401    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:33.748388    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:33.749162    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:33.749260    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:33.749334    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:33.749415    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:33.752762    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:33.752923    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:06:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:33.760672    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:34.126447    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:06:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:34.761796    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:35.153805    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:35.154111    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:35.761923    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:36.760841    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:37.761443    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:06:38.400332    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:38.400398    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:38.761281    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:39.765182    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:40.218218    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:40.218757    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:40.218867    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:40.218941    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:40.234412    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:40.347473    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:06:40Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-584d387968a2e117c6ee3188ea9d9db9d6c6d2ca90db768806a5b0a4f8d80827.scope/cpuset.cpus: permission denied"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:40.347569    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:06:40Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-584d387968a2e117c6ee3188ea9d9db9d6c6d2ca90db768806a5b0a4f8d80827.scope/cpuset.cpus: permission denied"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:40.347619    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:06:40Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-584d387968a2e117c6ee3188ea9d9db9d6c6d2ca90db768806a5b0a4f8d80827.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:40.753590    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:40.754334    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:40.754352    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:40.754360    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:40.754378    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:40.761498    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:40.761580    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:06:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:40.761637    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:41.761278    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:42.218002    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:42.218528    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:42.218626    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:42.218699    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:42.337584    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:06:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:06:42Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-3eefd1af37fad686e135995c36685b8d2a66a6f18780cdf3b20b15c21f385e88.scope/cpuset.cpus: permission denied"
Oct 25 19:06:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:06:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:42.337710    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:06:42Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-3eefd1af37fad686e135995c36685b8d2a66a6f18780cdf3b20b15c21f385e88.scope/cpuset.cpus: permission denied"
Oct 25 19:06:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:42.337765    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:06:42Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-3eefd1af37fad686e135995c36685b8d2a66a6f18780cdf3b20b15c21f385e88.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:06:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:42.760803    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:43.096371    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:43.096694    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:43.766033    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:44.127310    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:06:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:44.762167    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:44.895546    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:44.895577    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:45.761938    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:46.762233    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:47.762303    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:47.762399    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:47.763353    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:47.763376    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:47.763388    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:47.763426    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:47.766306    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:47.766380    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:06:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:48.761929    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:49.761379    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:50.239587    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:06:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:50.764110    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:06:51.684308    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:51.684369    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:51.765758    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:52.199754    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:52.199801    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:06:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:52.764622    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:53.217419    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:53.218142    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:53.218247    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:53.218348    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:53.340643    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:06:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:06:53Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7bdf919256dff9a1e2ebb6744e7f6061cab4a16577feef8b73329a53591b11d6.scope/cpuset.cpus: permission denied"
Oct 25 19:06:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:06:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:53.340741    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:06:53Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7bdf919256dff9a1e2ebb6744e7f6061cab4a16577feef8b73329a53591b11d6.scope/cpuset.cpus: permission denied"
Oct 25 19:06:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:53.340787    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:06:53Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7bdf919256dff9a1e2ebb6744e7f6061cab4a16577feef8b73329a53591b11d6.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:06:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:53.762093    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:54.128288    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:06:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:54.762616    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:54.767073    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:54.767715    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:54.767737    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:54.767748    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:54.767766    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:54.770676    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:06:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:54.770807    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:06:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:55.218000    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:06:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:55.218555    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:06:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:55.218651    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:06:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:55.218726    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:06:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:55.341462    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:06:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:06:55Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d0d4f6a2e3324ab1a9333ba29ab9fd96dc74746f33955890c7319858de4cae39.scope/cpuset.cpus: permission denied"
Oct 25 19:06:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:06:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:55.341575    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:06:55Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d0d4f6a2e3324ab1a9333ba29ab9fd96dc74746f33955890c7319858de4cae39.scope/cpuset.cpus: permission denied"
Oct 25 19:06:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:06:55.341634    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:06:55Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d0d4f6a2e3324ab1a9333ba29ab9fd96dc74746f33955890c7319858de4cae39.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:06:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:55.763902    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:56.760508    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:57.762252    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:58.763749    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:06:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:06:59.761670    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:00.245666    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925339748, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79de664": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:07:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:00.245751    4021 event.go:228] Unable to write event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa818096f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933344111, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}' (retry limit exceeded!)
Oct 25 19:07:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:00.249684    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:07:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:00.627472    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:00.627521    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:00.766847    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:01.762792    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:01.770988    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:01.771682    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:01.771726    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:01.771739    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:01.771756    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:01.775415    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:01.775437    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:07:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:02.764150    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:03.669670    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:03.669705    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:03.764109    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:04.129331    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:07:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:04.563741    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:07:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:04.765506    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:05.149624    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:05.149674    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:05.760731    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:06.762040    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:07.761512    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.217890    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.218222    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.218493    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.218520    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.218552    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.218904    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.218995    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.219075    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:08.368543    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-c1ee64eb16f57190f0f0148a0076c6d6cbb6b6a75b1dc4ce708d97709c9d13a3.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-c1ee64eb16f57190f0f0148a0076c6d6cbb6b6a75b1dc4ce708d97709c9d13a3.scope: no such file or directory
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:08.368976    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope: no such file or directory
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:08.369646    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/blkio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/blkio/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope: no such file or directory
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:08.369761    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/memory/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/memory/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope: no such file or directory
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:08.371019    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope: no such file or directory
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:08.371120    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/pids/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/pids/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope: no such file or directory
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:08.371143    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope: no such file or directory
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:08.397484    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:07:08Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope/cpuset.cpus: permission denied"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:08.397583    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:07:08Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope/cpuset.cpus: permission denied"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:08.397637    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:07:08Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8310ae6b3a02f87df774ecd378da6c7949f5490f59b6e0876f5722d46d01154a.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:08.413483    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:07:08Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-c1ee64eb16f57190f0f0148a0076c6d6cbb6b6a75b1dc4ce708d97709c9d13a3.scope/cpuset.cpus: permission denied"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:08.413562    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:07:08Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-c1ee64eb16f57190f0f0148a0076c6d6cbb6b6a75b1dc4ce708d97709c9d13a3.scope/cpuset.cpus: permission denied"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:08.413626    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:07:08Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-c1ee64eb16f57190f0f0148a0076c6d6cbb6b6a75b1dc4ce708d97709c9d13a3.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.762145    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.776366    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.777301    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.777318    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.777327    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:08.777346    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:08.785173    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:07:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:08.785175    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:09.759757    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:10.761681    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:11.763660    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:12.760847    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:13.100488    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:13.100537    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:13.761811    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:14.130748    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:07:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:14.569586    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:07:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:14.760741    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:14.895628    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:14.895747    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:15.763202    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:15.785252    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:15.785781    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:15.785868    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:15.785959    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:15.786061    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:15.790486    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:15.790501    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:07:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:16.763099    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:17.762226    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:18.761380    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:19.762675    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:20.217392    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:20.217949    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:20.217967    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:20.217977    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:20.340507    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:07:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:07:20Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-12ca353c41e9eb3b67a2dcdfc47538a8682c75a274060ccb063a36b7c26ae6c0.scope/cpuset.cpus: permission denied"
Oct 25 19:07:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:07:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:20.340799    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:07:20Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-12ca353c41e9eb3b67a2dcdfc47538a8682c75a274060ccb063a36b7c26ae6c0.scope/cpuset.cpus: permission denied"
Oct 25 19:07:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:20.340848    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:07:20Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-12ca353c41e9eb3b67a2dcdfc47538a8682c75a274060ccb063a36b7c26ae6c0.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:07:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:20.761844    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:21.218154    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:21.218985    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:21.219011    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:21.219022    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:21.356580    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:07:21Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-4258d127b492c6b9d88df9d02f55109ea665361fd0b09313a80397df4f83fffc.scope/cpuset.cpus: permission denied"
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:21.356737    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:07:21Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-4258d127b492c6b9d88df9d02f55109ea665361fd0b09313a80397df4f83fffc.scope/cpuset.cpus: permission denied"
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:21.356814    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:07:21Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-4258d127b492c6b9d88df9d02f55109ea665361fd0b09313a80397df4f83fffc.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:21.408403    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:21.408450    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:21.761722    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:21.914347    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:21.914464    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:22.198499    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:22.198649    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:22.764248    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:22.790635    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:22.791419    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:22.791441    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:22.791451    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:22.791469    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:22.797568    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:22.797623    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:07:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:23.761398    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:23.798795    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:07:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:23.799101    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:07:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:24.131507    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:07:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:24.574665    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:07:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:24.765031    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:25.762025    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:26.114008    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:26.114076    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:26.761143    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:27.761680    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:28.763145    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:29.760834    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:29.798209    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:29.799070    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:29.799110    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:29.799125    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:29.799145    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:29.802274    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:29.802409    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:07:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:30.629933    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:30.629984    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:30.761585    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:31.761653    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:32.761529    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:33.672724    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:33.672770    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:33.760697    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:34.131710    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:34.218139    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:34.218746    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:34.218848    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:34.218935    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:34.337526    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:07:34Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f5cf26ba5a13a141a90040e54c80d86fda8aa7b3f6a4f4d7bcff3279356cd946.scope/cpuset.cpus: permission denied"
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:34.337628    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:07:34Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f5cf26ba5a13a141a90040e54c80d86fda8aa7b3f6a4f4d7bcff3279356cd946.scope/cpuset.cpus: permission denied"
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:34.337676    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:07:34Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f5cf26ba5a13a141a90040e54c80d86fda8aa7b3f6a4f4d7bcff3279356cd946.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:34.580781    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:07:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:34.762038    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:35.149709    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:35.149757    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:35.217540    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:35.218104    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:35.218121    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:35.218131    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:35.313058    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a18c85eaa8c782f7e472775d44b3afb0af53ceac14c0b2c64a188f31881d20f5.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a18c85eaa8c782f7e472775d44b3afb0af53ceac14c0b2c64a188f31881d20f5.scope: no such file or directory
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:35.348125    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:07:35Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a18c85eaa8c782f7e472775d44b3afb0af53ceac14c0b2c64a188f31881d20f5.scope/cpuset.cpus: permission denied"
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:35.348217    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:07:35Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a18c85eaa8c782f7e472775d44b3afb0af53ceac14c0b2c64a188f31881d20f5.scope/cpuset.cpus: permission denied"
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:35.348269    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:07:35Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-a18c85eaa8c782f7e472775d44b3afb0af53ceac14c0b2c64a188f31881d20f5.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:07:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:35.762006    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:36.760623    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:36.802933    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:36.803542    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:36.803579    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:36.803592    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:36.803611    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:36.806648    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:36.806662    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:07:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:37.764953    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:38.762303    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:39.764890    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:40.761700    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:41.762393    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:42.769229    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:43.097755    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:43.097805    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:43.626514    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:43.626593    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:43.763240    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:43.807677    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:43.808339    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:43.808426    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:43.808496    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:43.808576    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:43.816829    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:43.816948    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:07:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:44.132056    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:07:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:44.584901    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:07:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:44.761925    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:44.892259    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:44.892600    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:45.217902    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:45.218391    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:45.218416    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:45.218427    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:45.337657    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:07:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:07:45Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-ea79e046a159c30c004975b52fc6c91ff4dfa4e9ecb658be71a013c2f7ff1ac3.scope/cpuset.cpus: permission denied"
Oct 25 19:07:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:07:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:45.337749    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:07:45Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-ea79e046a159c30c004975b52fc6c91ff4dfa4e9ecb658be71a013c2f7ff1ac3.scope/cpuset.cpus: permission denied"
Oct 25 19:07:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:45.337794    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:07:45Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-ea79e046a159c30c004975b52fc6c91ff4dfa4e9ecb658be71a013c2f7ff1ac3.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:07:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:45.760777    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:46.762133    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:47.760904    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:48.764691    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:49.217512    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:49.218257    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:49.218391    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:49.218462    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:07:49.312208    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-cdb32a316ea93765ec648d603da4e32dc194e6158c6f74a0251eebb6d2885b02.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-cdb32a316ea93765ec648d603da4e32dc194e6158c6f74a0251eebb6d2885b02.scope: no such file or directory
Oct 25 19:07:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:49.342715    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:07:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:07:49Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-cdb32a316ea93765ec648d603da4e32dc194e6158c6f74a0251eebb6d2885b02.scope/cpuset.cpus: permission denied"
Oct 25 19:07:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:07:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:49.342855    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:07:49Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-cdb32a316ea93765ec648d603da4e32dc194e6158c6f74a0251eebb6d2885b02.scope/cpuset.cpus: permission denied"
Oct 25 19:07:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:49.342920    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:07:49Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-cdb32a316ea93765ec648d603da4e32dc194e6158c6f74a0251eebb6d2885b02.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:07:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:49.762158    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:50.762380    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:50.817755    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:50.818592    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:50.818617    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:50.818629    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:50.818647    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:50.821764    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:50.821906    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:07:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:51.762754    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:52.198439    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:52.198491    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:07:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:52.766658    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:53.762524    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:54.133204    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:07:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:54.589282    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:07:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:54.766264    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:55.764497    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:56.762058    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:57.762974    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:57.821843    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:57.822895    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:57.822919    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:57.822931    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:57.822975    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:57.826822    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:07:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:57.826921    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:07:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:58.761079    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:07:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:59.217656    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:07:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:59.218183    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:07:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:59.218280    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:07:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:59.218346    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:07:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:59.339696    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:07:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:07:59Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-cf6d3580dd74d046542f87f1e428e949283041e9bbf72f5be33239952501fb51.scope/cpuset.cpus: permission denied"
Oct 25 19:07:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:07:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:59.339800    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:07:59Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-cf6d3580dd74d046542f87f1e428e949283041e9bbf72f5be33239952501fb51.scope/cpuset.cpus: permission denied"
Oct 25 19:07:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:07:59.339872    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:07:59Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-cf6d3580dd74d046542f87f1e428e949283041e9bbf72f5be33239952501fb51.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:07:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:07:59.761475    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:00.217379    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:00.217929    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:00.217948    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:00.217958    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:00.343588    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:08:00Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-6c40ada74f52f6be240066e5bad53154c4f3e33a9c2a6542c8aa919cda5b2445.scope/cpuset.cpus: permission denied"
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:00.343697    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:08:00Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-6c40ada74f52f6be240066e5bad53154c4f3e33a9c2a6542c8aa919cda5b2445.scope/cpuset.cpus: permission denied"
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:00.343768    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:08:00Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-6c40ada74f52f6be240066e5bad53154c4f3e33a9c2a6542c8aa919cda5b2445.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:00.626517    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:00.626564    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:00.759398    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:01.761186    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:02.762446    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:03.670523    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:03.670571    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:03.761630    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:04.133580    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:08:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:04.594639    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:08:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:04.763579    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:04.827505    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:04.828156    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:04.828173    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:04.828186    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:04.828203    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:04.832444    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:04.832515    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:08:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:08:05.019495    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:05.019621    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:05.150234    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:05.150262    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:05.762151    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:06.762965    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:07.765633    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:08.763019    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:09.759368    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:10.217924    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:10.218651    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:10.218824    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:10.218942    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:10.335339    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:08:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:08:10Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a20c97e730c7cd335c651e76b6fd2454da442f067abcfcfeb82f877a29109f23.scope/cpuset.cpus: permission denied"
Oct 25 19:08:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:08:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:10.335446    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:08:10Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a20c97e730c7cd335c651e76b6fd2454da442f067abcfcfeb82f877a29109f23.scope/cpuset.cpus: permission denied"
Oct 25 19:08:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:10.335504    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:08:10Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-a20c97e730c7cd335c651e76b6fd2454da442f067abcfcfeb82f877a29109f23.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:08:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:10.764841    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:11.218012    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:11.218605    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:11.218696    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:11.218766    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:08:11.307659    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-64342c58cd4ff380c229cdad7b2db3efd9b135b548561aa5d986e5e18395012d.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-64342c58cd4ff380c229cdad7b2db3efd9b135b548561aa5d986e5e18395012d.scope: no such file or directory
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:11.336892    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:08:11Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-64342c58cd4ff380c229cdad7b2db3efd9b135b548561aa5d986e5e18395012d.scope/cpuset.cpus: permission denied"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:11.337021    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:08:11Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-64342c58cd4ff380c229cdad7b2db3efd9b135b548561aa5d986e5e18395012d.scope/cpuset.cpus: permission denied"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:11.337084    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:08:11Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-64342c58cd4ff380c229cdad7b2db3efd9b135b548561aa5d986e5e18395012d.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:11.762436    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:11.832841    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:11.833597    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:11.833618    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:11.833627    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:11.833665    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:11.838043    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:11.838166    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:08:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:12.764723    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:13.102245    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:13.102274    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:08:13.237904    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:13.237948    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:13.762904    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:14.134863    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:08:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:14.651765    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:08:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:14.760708    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:14.893595    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:14.893645    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:15.764911    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:16.760937    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:17.760965    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:18.762655    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:18.838729    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:18.839395    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:18.839419    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:18.839456    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:18.839476    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:18.842895    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:08:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:18.842877    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:19.762616    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:20.764903    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:08:21.196171    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:21.196218    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:21.761881    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:08:21.782977    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:21.783028    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:22.199999    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:22.200113    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:22.218222    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:22.218301    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:22.218841    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:22.218929    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:22.219016    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:22.219185    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:22.219226    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:22.219237    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:08:22.352921    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7ab46e0e8dff88e7abd64ff482c7490021a7d6e4b2d60d62d18d5a5804c06938.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/devices/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7ab46e0e8dff88e7abd64ff482c7490021a7d6e4b2d60d62d18d5a5804c06938.scope: no such file or directory
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:08:22.353076    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/pids/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7ab46e0e8dff88e7abd64ff482c7490021a7d6e4b2d60d62d18d5a5804c06938.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/pids/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7ab46e0e8dff88e7abd64ff482c7490021a7d6e4b2d60d62d18d5a5804c06938.scope: no such file or directory
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:08:22.357549    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7ab46e0e8dff88e7abd64ff482c7490021a7d6e4b2d60d62d18d5a5804c06938.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7ab46e0e8dff88e7abd64ff482c7490021a7d6e4b2d60d62d18d5a5804c06938.scope: no such file or directory
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:22.390008    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:08:22Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-8ec6a836ffc412c2206cddc40fbcdaf5329f4094aa6b586dd1f6415b55c31fb9.scope/cpuset.cpus: permission denied"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:22.390111    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:08:22Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-8ec6a836ffc412c2206cddc40fbcdaf5329f4094aa6b586dd1f6415b55c31fb9.scope/cpuset.cpus: permission denied"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:22.390165    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:08:22Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-8ec6a836ffc412c2206cddc40fbcdaf5329f4094aa6b586dd1f6415b55c31fb9.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:22.405318    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:08:22Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7ab46e0e8dff88e7abd64ff482c7490021a7d6e4b2d60d62d18d5a5804c06938.scope/cpuset.cpus: permission denied"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:22.405396    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:08:22Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7ab46e0e8dff88e7abd64ff482c7490021a7d6e4b2d60d62d18d5a5804c06938.scope/cpuset.cpus: permission denied"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:22.405471    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:08:22Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-7ab46e0e8dff88e7abd64ff482c7490021a7d6e4b2d60d62d18d5a5804c06938.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:08:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:22.762030    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:23.762324    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:23.799800    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:08:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:23.799844    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:08:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:24.135082    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:08:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:24.659801    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:08:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:24.761534    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:25.762845    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:25.843816    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:25.844354    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:25.844441    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:25.844522    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:25.844602    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:25.848421    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:25.848512    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:08:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:26.761652    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:27.761303    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:28.760523    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:29.761712    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:30.626801    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:30.626869    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:30.761753    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:31.761783    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:32.760989    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:32.849627    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:32.850227    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:32.850265    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:32.850276    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:32.850295    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:32.854739    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:32.854752    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:33.218003    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:33.218649    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:33.218735    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:33.218805    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:33.344680    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:08:33Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d3d9bb46cf7f384c22dc2af12a1dc5bb63b8f5d41914fe0b46f8121d61ce3f40.scope/cpuset.cpus: permission denied"
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:33.344806    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:08:33Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d3d9bb46cf7f384c22dc2af12a1dc5bb63b8f5d41914fe0b46f8121d61ce3f40.scope/cpuset.cpus: permission denied"
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:33.344862    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:08:33Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d3d9bb46cf7f384c22dc2af12a1dc5bb63b8f5d41914fe0b46f8121d61ce3f40.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:33.674962    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:33.675013    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:33.761402    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:34.135766    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:08:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:34.663719    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:08:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:34.761102    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:35.153731    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:35.153778    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:35.763646    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:36.762673    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:37.217766    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:37.218382    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:37.218519    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:37.218604    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:37.339626    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:08:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:08:37Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f27dfaaa159ba0ca51ce721ea9a4618763167746b83ebbd2b07bea9b60fd6eff.scope/cpuset.cpus: permission denied"
Oct 25 19:08:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:08:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:37.339735    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:08:37Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f27dfaaa159ba0ca51ce721ea9a4618763167746b83ebbd2b07bea9b60fd6eff.scope/cpuset.cpus: permission denied"
Oct 25 19:08:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:37.339789    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:08:37Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f27dfaaa159ba0ca51ce721ea9a4618763167746b83ebbd2b07bea9b60fd6eff.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:08:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:37.762201    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:38.762651    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:39.761259    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:39.855442    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:39.856101    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:39.856143    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:39.856155    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:39.856174    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:39.859196    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:39.859209    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:08:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:40.763715    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:41.760630    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:42.763810    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:43.095984    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:43.096137    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:43.760906    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:44.137057    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:08:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:44.671019    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925351978, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e162a": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:08:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:44.671210    4021 event.go:228] Unable to write event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa8182d05", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933353221, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}' (retry limit exceeded!)
Oct 25 19:08:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:44.677411    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:08:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:44.765374    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:44.898798    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:44.898842    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:45.762272    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:46.217937    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:46.218494    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:46.218589    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:46.218663    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:46.340597    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:08:46Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-7255c8e4f8c268573ca7a3225030048227a878c69e11b42a3a0203a80aad971c.scope/cpuset.cpus: permission denied"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:46.340689    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:08:46Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-7255c8e4f8c268573ca7a3225030048227a878c69e11b42a3a0203a80aad971c.scope/cpuset.cpus: permission denied"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:46.340734    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:08:46Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-7255c8e4f8c268573ca7a3225030048227a878c69e11b42a3a0203a80aad971c.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:46.761593    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:46.859464    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:46.860173    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:46.860199    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:46.860210    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:46.860247    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:46.864414    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:08:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:46.864421    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:47.762288    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:08:47.927302    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:47.927360    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:48.761322    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:49.761625    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:50.217940    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:50.218517    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:50.218586    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:50.218600    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:50.333601    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:08:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:08:50Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-376d682f989705f8a257410485913338f06ea28e6755bcb81fdbe2639db7d4e2.scope/cpuset.cpus: permission denied"
Oct 25 19:08:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:08:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:50.333693    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:08:50Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-376d682f989705f8a257410485913338f06ea28e6755bcb81fdbe2639db7d4e2.scope/cpuset.cpus: permission denied"
Oct 25 19:08:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:50.333739    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:08:50Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-376d682f989705f8a257410485913338f06ea28e6755bcb81fdbe2639db7d4e2.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:08:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:50.763779    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:51.761797    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:52.198574    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:52.198894    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:08:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:52.766705    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:08:52.847676    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:52.847840    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:53.765207    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:53.864845    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:08:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:53.865423    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:08:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:53.865511    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:08:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:53.865581    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:08:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:53.865654    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:53.870450    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:08:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:53.870465    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:08:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:54.138056    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:08:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:08:54.411944    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:08:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:54.761340    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:55.760498    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:56.762170    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:57.766143    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:58.761238    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:08:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:08:59.768512    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:09:00.098273    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:00.098415    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:00.629327    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:00.629491    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:00.763524    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:00.870908    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:00.871990    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:00.872013    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:00.872026    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:00.872067    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:00.874867    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:00.874953    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:01.217262    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:01.217309    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:01.217940    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:01.217957    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:01.218221    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:01.218170    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:01.218383    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:01.218395    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:01.385950    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:09:01Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-36872ed76d2f4422d6e66470a9ef5fd032d6141dc4fcbc793a2b8569979c2afe.scope/cpuset.cpus: permission denied"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:01.386063    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:09:01Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-36872ed76d2f4422d6e66470a9ef5fd032d6141dc4fcbc793a2b8569979c2afe.scope/cpuset.cpus: permission denied"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:01.386116    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:09:01Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-36872ed76d2f4422d6e66470a9ef5fd032d6141dc4fcbc793a2b8569979c2afe.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:01.401540    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:09:01Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-6fa690220a6eb61d87117f9bb4e1eeaae4e34862de1dad412eb8bd2f3292de8f.scope/cpuset.cpus: permission denied"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:01.401592    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:09:01Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-6fa690220a6eb61d87117f9bb4e1eeaae4e34862de1dad412eb8bd2f3292de8f.scope/cpuset.cpus: permission denied"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:01.401625    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:09:01Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-6fa690220a6eb61d87117f9bb4e1eeaae4e34862de1dad412eb8bd2f3292de8f.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:09:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:01.760814    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:02.762555    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:03.670208    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:03.670352    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:03.761808    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:04.138921    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:09:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:04.416284    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:09:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:09:04.600585    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:04.600643    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:04.762885    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:05.148803    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:05.148831    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:05.761367    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:06.761139    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:07.766677    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:07.875326    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:07.876122    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:07.876218    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:07.876290    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:07.876372    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:07.880603    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:07.880615    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:09:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:08.762209    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:09.761968    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:10.762002    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:11.765517    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:12.761295    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:13.098321    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:13.098376    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:13.217519    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:13.218009    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:13.218103    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:13.218170    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:13.336902    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:09:13Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-202ce3067d3bee4af1fcca594cccd16c3af5574a2db1f524f808ac0127fa6247.scope/cpuset.cpus: permission denied"
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:13.336999    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:09:13Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-202ce3067d3bee4af1fcca594cccd16c3af5574a2db1f524f808ac0127fa6247.scope/cpuset.cpus: permission denied"
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:13.337043    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:09:13Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-202ce3067d3bee4af1fcca594cccd16c3af5574a2db1f524f808ac0127fa6247.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:09:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:13.761187    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:14.139797    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:14.423055    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:14.765174    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:14.881452    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:14.882131    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:14.882225    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:14.882297    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:14.882377    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:14.889723    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:14.889919    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:14.889921    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:14.889866    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:09:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:15.761067    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:16.217838    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:16.218529    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:16.218550    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:16.218562    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:16.363070    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:09:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:09:16Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f1ec9a626ef51adefd2603dd29431ec562f8c23066f37905a3fb9a49cab14053.scope/cpuset.cpus: permission denied"
Oct 25 19:09:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:09:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:16.364901    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:09:16Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f1ec9a626ef51adefd2603dd29431ec562f8c23066f37905a3fb9a49cab14053.scope/cpuset.cpus: permission denied"
Oct 25 19:09:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:16.365066    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:09:16Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-f1ec9a626ef51adefd2603dd29431ec562f8c23066f37905a3fb9a49cab14053.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:09:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:16.763810    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:17.761728    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:18.760787    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:19.760750    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:20.762280    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:21.757726    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:21.890686    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:21.891716    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:21.891805    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:21.891905    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:21.891986    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:21.896160    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:21.896176    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:09:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:22.200482    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:22.200533    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:22.761849    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:23.762469    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:23.800564    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:09:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:23.800678    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:09:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:24.140724    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:09:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:24.427753    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:09:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:24.762552    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:25.762126    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:26.762780    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:27.217896    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:27.217901    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:27.218596    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:27.218714    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:27.218792    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:27.218805    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:27.218758    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:27.219066    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:27.409796    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:09:27Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-63af891cee9d26569f0c12ecb0c9d775dfc3c66f84d63dd6ccf90f50f1762347.scope/cpuset.cpus: permission denied"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:27.409922    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:09:27Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-63af891cee9d26569f0c12ecb0c9d775dfc3c66f84d63dd6ccf90f50f1762347.scope/cpuset.cpus: permission denied"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:27.409985    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:09:27Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-63af891cee9d26569f0c12ecb0c9d775dfc3c66f84d63dd6ccf90f50f1762347.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:27.425454    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:09:27Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8a640001602dc85274be3e9e0de5e43a191d8258956a82b29849d2347bb7be26.scope/cpuset.cpus: permission denied"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:27.425530    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:09:27Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8a640001602dc85274be3e9e0de5e43a191d8258956a82b29849d2347bb7be26.scope/cpuset.cpus: permission denied"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:27.425567    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:09:27Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8a640001602dc85274be3e9e0de5e43a191d8258956a82b29849d2347bb7be26.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:09:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:27.762722    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:28.762985    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:28.897304    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:28.898066    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:28.898174    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:28.898264    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:28.898367    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:28.901337    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:28.901476    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:09:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:29.764608    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:30.627197    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:30.627244    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:30.761228    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:31.761683    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:32.762136    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:33.672362    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:33.672477    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:33.761036    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:34.141228    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:09:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:34.432590    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:09:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:34.762481    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:35.152419    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:35.152466    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:35.761511    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:35.901991    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:35.902749    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:35.902917    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:35.903009    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:35.903089    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:35.905548    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:35.905621    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:09:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:36.761476    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:37.762427    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:38.218063    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:38.218623    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:38.218712    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:38.218789    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:38.341522    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:09:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:09:38Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-050c21c9447136318b593d45296e9835c6089601f68061c8be4d9024dc502ce0.scope/cpuset.cpus: permission denied"
Oct 25 19:09:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:09:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:38.341623    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:09:38Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-050c21c9447136318b593d45296e9835c6089601f68061c8be4d9024dc502ce0.scope/cpuset.cpus: permission denied"
Oct 25 19:09:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:38.341671    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:09:38Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-050c21c9447136318b593d45296e9835c6089601f68061c8be4d9024dc502ce0.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:09:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:38.764206    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:09:39.574925    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:39.574987    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:39.763666    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:09:40.008267    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:40.008303    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:40.217859    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:40.218435    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:40.218529    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:40.218601    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:40.341586    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:09:40Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-7b11b11726f87c298d30b600459a67fa94ed510045d3763ce3489d9a9edac7a8.scope/cpuset.cpus: permission denied"
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:40.341689    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:09:40Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-7b11b11726f87c298d30b600459a67fa94ed510045d3763ce3489d9a9edac7a8.scope/cpuset.cpus: permission denied"
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:40.341746    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:09:40Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-7b11b11726f87c298d30b600459a67fa94ed510045d3763ce3489d9a9edac7a8.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:09:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:40.762352    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:41.761090    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:42.761148    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:42.905925    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:42.906666    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:42.906766    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:42.906867    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:42.906971    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:42.913480    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:42.913596    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:09:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:43.097933    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:43.097966    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:09:43.751579    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:43.751990    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:43.761006    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:44.141461    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:09:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:44.437075    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:09:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:44.762875    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:44.895404    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:44.895518    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:45.768790    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:46.762415    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:47.761955    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:48.760973    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:49.764302    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:49.913914    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:49.914470    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:49.914495    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:49.914507    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:49.914525    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:49.918031    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:49.918169    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:09:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:50.764660    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:51.217430    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:51.217965    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:51.217982    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:51.217991    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:09:51.243342    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:51.243537    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:51.351665    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:09:51Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-8b5916045c0d95686713f0e4079b1c4d7e461764aca67e156b2177c85a911a0a.scope/cpuset.cpus: permission denied"
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:51.351768    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:09:51Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-8b5916045c0d95686713f0e4079b1c4d7e461764aca67e156b2177c85a911a0a.scope/cpuset.cpus: permission denied"
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:51.351838    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:09:51Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-8b5916045c0d95686713f0e4079b1c4d7e461764aca67e156b2177c85a911a0a.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:09:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:51.761214    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:52.199550    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:52.199995    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:09:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:52.761104    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:53.218075    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:53.218784    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:53.218801    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:53.218846    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:53.349040    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:09:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:09:53Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e0d8a6bec73f47cbde3b448f8a64152aead35bcc68d2b475bb7aa5c38a40e705.scope/cpuset.cpus: permission denied"
Oct 25 19:09:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:09:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:53.349150    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:09:53Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e0d8a6bec73f47cbde3b448f8a64152aead35bcc68d2b475bb7aa5c38a40e705.scope/cpuset.cpus: permission denied"
Oct 25 19:09:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:53.349213    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:09:53Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e0d8a6bec73f47cbde3b448f8a64152aead35bcc68d2b475bb7aa5c38a40e705.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:09:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:53.763243    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:54.141533    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:09:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:54.442210    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:09:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:54.761958    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:55.766148    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:56.762955    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:56.919154    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:09:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:56.920125    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:09:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:56.920162    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:09:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:56.920175    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:09:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:56.920204    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:56.925610    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:09:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:09:56.925647    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:09:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:57.761079    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:58.763062    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:09:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:09:59.764657    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:00.626785    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:00.626949    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:00.764385    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:01.762627    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:02.760957    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:03.218212    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:03.219048    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:03.219096    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:03.219111    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:03.348730    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:10:03Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-cf590a313f0dc07e21bcd49f85cf69991239209c745a78de6ec3c47b4ac50de7.scope/cpuset.cpus: permission denied"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:03.348830    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:10:03Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-cf590a313f0dc07e21bcd49f85cf69991239209c745a78de6ec3c47b4ac50de7.scope/cpuset.cpus: permission denied"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:03.348899    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:10:03Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-cf590a313f0dc07e21bcd49f85cf69991239209c745a78de6ec3c47b4ac50de7.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:03.667336    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:03.667366    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:03.764273    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:03.926283    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:03.927108    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:03.927197    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:03.927268    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:03.927355    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:03.933693    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:03.933840    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:10:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:04.141609    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:10:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:04.447720    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:10:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:04.762435    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:05.154674    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:05.154801    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:05.763075    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:06.218201    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:06.218649    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:06.218739    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:06.218812    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:10:06.314005    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c8300b86016fe34637684039adf9eef3d4f04a411c5d13ba4c464c91589551ee.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c8300b86016fe34637684039adf9eef3d4f04a411c5d13ba4c464c91589551ee.scope: no such file or directory
Oct 25 19:10:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:06.337649    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:10:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:10:06Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c8300b86016fe34637684039adf9eef3d4f04a411c5d13ba4c464c91589551ee.scope/cpuset.cpus: permission denied"
Oct 25 19:10:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:10:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:06.337745    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:10:06Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c8300b86016fe34637684039adf9eef3d4f04a411c5d13ba4c464c91589551ee.scope/cpuset.cpus: permission denied"
Oct 25 19:10:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:06.337800    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:10:06Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c8300b86016fe34637684039adf9eef3d4f04a411c5d13ba4c464c91589551ee.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:10:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:06.764970    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:07.761864    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:08.762104    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:09.761775    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:10.762530    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:10:10.803440    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:10.803589    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:10.934607    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:10.935238    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:10.935346    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:10.935444    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:10.935539    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:10.941510    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:10:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:10.941512    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:11.760710    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:12.764283    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:13.098445    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:13.098597    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:13.763705    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:14.142675    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:10:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:14.455261    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:10:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:14.761787    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:14.891779    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:14.891835    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:15.217685    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:15.218225    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:15.218316    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:15.218384    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:15.344568    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:10:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:10:15Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-bd00d497a82e81c2e3195ddc753406158b097db72deb2fd68a0902a5e7cffc52.scope/cpuset.cpus: permission denied"
Oct 25 19:10:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:10:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:15.344665    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:10:15Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-bd00d497a82e81c2e3195ddc753406158b097db72deb2fd68a0902a5e7cffc52.scope/cpuset.cpus: permission denied"
Oct 25 19:10:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:15.344717    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:10:15Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-bd00d497a82e81c2e3195ddc753406158b097db72deb2fd68a0902a5e7cffc52.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:10:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:15.765353    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:16.761209    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:10:17.711646    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:17.711841    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:17.761850    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:17.942336    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:17.943159    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:17.943185    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:17.943197    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:17.943219    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:17.946208    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:17.946327    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:10:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:18.760948    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:19.761592    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:20.758929    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:21.765788    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:22.198441    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:22.198576    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:22.217612    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:22.218134    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:22.218240    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:22.218328    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:10:22.312943    4021 watcher.go:93] Error while processing event ("/sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c9d933577e99985b2cbe805a8eb045a2ec40d02341fcc960ccb7796684e91211.scope": 0x40000100 == IN_CREATE|IN_ISDIR): readdirent /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c9d933577e99985b2cbe805a8eb045a2ec40d02341fcc960ccb7796684e91211.scope: no such file or directory
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:22.340752    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:10:22Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c9d933577e99985b2cbe805a8eb045a2ec40d02341fcc960ccb7796684e91211.scope/cpuset.cpus: permission denied"
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:22.340894    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:10:22Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c9d933577e99985b2cbe805a8eb045a2ec40d02341fcc960ccb7796684e91211.scope/cpuset.cpus: permission denied"
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:22.340949    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:10:22Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-c9d933577e99985b2cbe805a8eb045a2ec40d02341fcc960ccb7796684e91211.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:22.762519    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:10:22.888829    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:22.888877    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:23.761293    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:23.801598    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:10:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:23.801732    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:10:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:24.143057    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:10:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:24.459381    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:10:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:24.761995    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:24.947064    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:24.947786    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:24.947809    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:24.947820    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:24.947841    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:24.951376    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:24.951431    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:10:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:25.761123    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:26.761361    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:27.218170    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:27.218655    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:27.218751    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:27.218815    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:27.337710    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:10:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:10:27Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-8e93b59fa5bef27c3f3f43f976888983c61e02897f40c80941b5b7ae7b579483.scope/cpuset.cpus: permission denied"
Oct 25 19:10:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:10:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:27.337809    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:10:27Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-8e93b59fa5bef27c3f3f43f976888983c61e02897f40c80941b5b7ae7b579483.scope/cpuset.cpus: permission denied"
Oct 25 19:10:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:27.337872    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:10:27Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-8e93b59fa5bef27c3f3f43f976888983c61e02897f40c80941b5b7ae7b579483.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:10:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:27.762515    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:28.762576    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:29.761533    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:30.629865    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:30.630306    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:30.764529    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:31.761500    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:31.951871    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:31.952566    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:31.952658    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:31.952739    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:31.952827    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:31.956311    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:31.956443    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:10:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:32.762932    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:33.669988    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:33.670126    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:33.761953    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:34.143695    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:10:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:34.464776    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 925360799, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:2, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Patch "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa79e389f": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:10:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:34.465000    4021 event.go:228] Unable to write event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfa818494e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node nndncs4w-b3ed8-rw8t5-worker-0-bbpcn status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 23, 933360462, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}' (retry limit exceeded!)
Oct 25 19:10:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:34.470149    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfb1d0cf98", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:10:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:34.761331    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:35.150107    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:35.150154    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:35.217693    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:35.218283    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:35.218366    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:35.218444    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:35.332517    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:10:35Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-35853828d4e930a7fcee3ef8a3c5f22b9334473ae5d691f58c8dd105d1a99691.scope/cpuset.cpus: permission denied"
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:35.332624    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:10:35Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-35853828d4e930a7fcee3ef8a3c5f22b9334473ae5d691f58c8dd105d1a99691.scope/cpuset.cpus: permission denied"
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:35.332676    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:10:35Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-35853828d4e930a7fcee3ef8a3c5f22b9334473ae5d691f58c8dd105d1a99691.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:10:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:35.763615    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:36.761508    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:37.242516    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfb1d0cf98", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:10:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:37.762233    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:38.764516    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:38.957287    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:38.957963    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:38.958005    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:38.958019    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:38.958038    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:38.961365    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:38.961395    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:10:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:39.763409    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:40.758329    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:41.761401    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:42.217349    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:42.217946    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:42.218042    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:42.218120    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:42.332507    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:10:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:10:42Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-22e4eede82bfb060c77ebcf36ee1f9e0f32af1f81b995d272e50ba5e866fa72e.scope/cpuset.cpus: permission denied"
Oct 25 19:10:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:10:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:42.332616    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:10:42Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-22e4eede82bfb060c77ebcf36ee1f9e0f32af1f81b995d272e50ba5e866fa72e.scope/cpuset.cpus: permission denied"
Oct 25 19:10:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:42.332674    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:10:42Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-22e4eede82bfb060c77ebcf36ee1f9e0f32af1f81b995d272e50ba5e866fa72e.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:10:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:42.761900    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:43.095983    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:43.096107    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:10:43.353353    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:43.353500    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:43.761431    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:44.143862    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:10:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:44.762281    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:44.892913    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:44.892942    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:45.761545    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:45.962361    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:45.962971    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:45.963061    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:45.963130    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:45.963211    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:45.964916    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:45.965012    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:10:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:46.765115    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:47.217929    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:47.218431    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:47.218525    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:47.218608    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:47.251240    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfb1d0cf98", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:10:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:47.337559    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:10:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:10:47Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8fc446a4d1386078bdb35d2f249773202a3e4521b3c0cc52bb9b44d4d0185e04.scope/cpuset.cpus: permission denied"
Oct 25 19:10:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:10:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:47.337656    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:10:47Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8fc446a4d1386078bdb35d2f249773202a3e4521b3c0cc52bb9b44d4d0185e04.scope/cpuset.cpus: permission denied"
Oct 25 19:10:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:47.337707    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:10:47Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-8fc446a4d1386078bdb35d2f249773202a3e4521b3c0cc52bb9b44d4d0185e04.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:10:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:47.761147    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:48 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:48.764355    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:49 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:49.761022    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:50 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:50.761572    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:10:51.667863    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:51.668070    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:51 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:51.761516    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:52.197586    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:52.197619    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:10:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:52.764544    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:52.965861    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:52.966733    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:52.966833    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:52.966929    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:52.967023    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:52.970209    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:52 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:52.970335    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:10:53 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:53.764595    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:54.144997    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:10:54 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:54.766454    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:55 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:55.765555    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:56 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:56.765394    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:57.217730    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:57.218455    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:57.218543    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:57.218615    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:57.256499    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfb1d0cf98", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:10:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:57.347649    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:10:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:10:57Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-f3a370293c6ab1998d608df8b8c821fe98735d993307fdab28248682c9fd3aca.scope/cpuset.cpus: permission denied"
Oct 25 19:10:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:10:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:57.347755    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:10:57Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-f3a370293c6ab1998d608df8b8c821fe98735d993307fdab28248682c9fd3aca.scope/cpuset.cpus: permission denied"
Oct 25 19:10:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:57.347808    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:10:57Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-f3a370293c6ab1998d608df8b8c821fe98735d993307fdab28248682c9fd3aca.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:10:57 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:57.763481    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:58 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:58.762029    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:59.762183    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:10:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:59.971257    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:10:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:59.972187    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:10:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:59.972281    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:10:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:59.972358    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:10:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:10:59.972438    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:59.975251    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:10:59 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:10:59.975376    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:11:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:00.629303    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:00.629350    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:11:00 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:00.761918    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:01.217469    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:01.218022    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:01.218041    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:01.218052    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:01.347826    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:11:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:11:01Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e20130ba26eef860392f45a575d6f904e8713d5a24ac58989243bc4d9d726bba.scope/cpuset.cpus: permission denied"
Oct 25 19:11:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:11:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:01.347930    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:11:01Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e20130ba26eef860392f45a575d6f904e8713d5a24ac58989243bc4d9d726bba.scope/cpuset.cpus: permission denied"
Oct 25 19:11:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:01.347975    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:11:01Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-e20130ba26eef860392f45a575d6f904e8713d5a24ac58989243bc4d9d726bba.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:11:01 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:01.794203    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:02 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:02.764465    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:03.670501    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:03.670634    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:11:03 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:03.762496    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:04.145065    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:11:04 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:04.761160    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:05.152732    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:05.152760    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:11:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:05.765647    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:11:05.773634    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:05 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:05.773780    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes?fieldSelector=metadata.name%3Dnndncs4w-b3ed8-rw8t5-worker-0-bbpcn&limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:06.762381    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:06.976223    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:06.976949    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:06.976968    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:06.976979    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:06.976997    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:06.980303    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:11:06 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:06.980314    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:07.262241    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfb1d0cf98", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:11:07 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:07.761618    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:08 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:08.761747    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:09 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:09.761476    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:10 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:10.763690    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:11 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:11.762420    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:12.217469    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:12.218213    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:12.218232    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:12.218242    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:12.343631    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:11:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:11:12Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d7f21176e226004a1467b69e8101eb99f08eaad1563c5d2857e66a8aeaaed3fc.scope/cpuset.cpus: permission denied"
Oct 25 19:11:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:11:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:12.343773    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:11:12Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d7f21176e226004a1467b69e8101eb99f08eaad1563c5d2857e66a8aeaaed3fc.scope/cpuset.cpus: permission denied"
Oct 25 19:11:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:12.343836    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:11:12Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d7f21176e226004a1467b69e8101eb99f08eaad1563c5d2857e66a8aeaaed3fc.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:11:12 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:12.779478    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:13.098046    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:13.098200    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:11:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:13.761009    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:13.980964    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:13.981731    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:13.981822    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:13.981914    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:13.981993    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:13.984954    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:13 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:13.985077    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:11:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:14.146137    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:11:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:14.762658    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:14.902516    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:14 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:14.902630    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:11:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:15.217765    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:15.218397    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:15.218500    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:15.218576    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:15.344640    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:11:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:11:15Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-efa5f8ea91a4c8e7dd5fe2a4926042e789a0a63b7f5c947034f648beb3689aa8.scope/cpuset.cpus: permission denied"
Oct 25 19:11:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:11:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:15.344760    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:11:15Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-efa5f8ea91a4c8e7dd5fe2a4926042e789a0a63b7f5c947034f648beb3689aa8.scope/cpuset.cpus: permission denied"
Oct 25 19:11:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:15.344814    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:11:15Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-efa5f8ea91a4c8e7dd5fe2a4926042e789a0a63b7f5c947034f648beb3689aa8.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:11:15 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:15.762623    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:16 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:16.768231    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:17.270086    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfb1d0cf98", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:11:17 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:17.761600    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:18 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:18.763381    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:19 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:19.760923    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:20.765491    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:11:20.779339    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:20.779467    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:20.985562    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:20.986336    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:20.986355    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:20.986366    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:20.986403    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:20.992153    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:20 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:20.992218    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:11:21 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:21.762697    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:22.197265    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:22.197725    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:11:22 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:22.761925    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:23.760747    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:23.801988    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:11:23 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:23.802101    4021 kubelet_getters.go:187] "Pod status updated" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" status="Pending"
Oct 25 19:11:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:24.147049    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:11:24 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:24.765050    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:25 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:25.764700    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:26 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:26.762565    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:27.217412    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:27.217948    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:27.217983    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:27.217995    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:27.275893    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfb1d0cf98", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:27.343039    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:11:27Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-acf1fb88d6da31e741063cb315eb1dbef6f530066a5f0e03721dda573a11ec13.scope/cpuset.cpus: permission denied"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:27.343137    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:11:27Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-acf1fb88d6da31e741063cb315eb1dbef6f530066a5f0e03721dda573a11ec13.scope/cpuset.cpus: permission denied"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:27.343190    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:11:27Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-acf1fb88d6da31e741063cb315eb1dbef6f530066a5f0e03721dda573a11ec13.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:27.762087    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:27.992520    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:27.993355    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:27.993394    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:27.993405    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:27.993426    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:27.996524    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:27 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:27.996578    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:11:28 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:28.765570    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:29 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:29.762092    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:30.217576    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:30.218147    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:30.218170    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:30.218181    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:30.338735    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:11:30Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-57aca71d78974557479833a715b171d5f4d30a7822b2229f7fe45c592bf478f0.scope/cpuset.cpus: permission denied"
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:30.338857    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:11:30Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-57aca71d78974557479833a715b171d5f4d30a7822b2229f7fe45c592bf478f0.scope/cpuset.cpus: permission denied"
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:30.338925    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:11:30Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-57aca71d78974557479833a715b171d5f4d30a7822b2229f7fe45c592bf478f0.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:30.625904    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:30.625934    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:11:30 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:30.763786    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:31 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:31.761110    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:32 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:32.761484    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:33.672425    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:33.672475    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:11:33 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:33.762444    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:34.147532    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:11:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:34.761396    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:34.996684    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:34.997490    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:34.997509    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:34.997520    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:34 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:34.997555    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:35.002137    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:35.002213    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:11:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:35.153034    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:35.153063    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:11:35 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:35.761643    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:36 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:36.761654    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:37.280648    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfb1d0cf98", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)
Oct 25 19:11:37 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:37.764955    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:38 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:38.762333    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:39 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:39.761328    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:40.762483    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:11:40.816565    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:40 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:40.816617    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/services?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: W1025 19:11:41.012578    4021 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:41.012725    4021 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:41.217902    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:41.218566    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:41.218737    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:41.218919    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:41.343558    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:11:41Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d7642564360a3155e7defb4ba4d426ed1b712d7d6d8e3374b66d7153f1e7d3e2.scope/cpuset.cpus: permission denied"
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="d211ad132ad1ddbdfdd6a4e51cee6bbaee87d801913d7ad8fe2e545721e92f8f"
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:41.343897    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-coredns,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /var/lib/kubelet/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/coredns --resolvconf-path /var/run/NetworkManager/resolv.conf],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/var/lib/kubelet,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:resource-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/coredns,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nm-resolv,ReadOnly:false,MountPath:/var/run/NetworkManager,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(74b8c21149c0164c4da6a5858bca8c45): CreateContainerError: container create failed: time="2023-10-25T19:11:41Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d7642564360a3155e7defb4ba4d426ed1b712d7d6d8e3374b66d7153f1e7d3e2.scope/cpuset.cpus: permission denied"
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:41.343950    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-coredns\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:11:41Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b8c21149c0164c4da6a5858bca8c45.slice/crio-d7642564360a3155e7defb4ba4d426ed1b712d7d6d8e3374b66d7153f1e7d3e2.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/coredns-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="74b8c21149c0164c4da6a5858bca8c45"
Oct 25 19:11:41 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:41.761552    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:42.002774    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:42.003617    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:42.003639    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:42.003649    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:42.003669    4021 kubelet_node_status.go:72] "Attempting to register node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:42.007270    4021 kubelet_node_status.go:94] "Unable to register node with API server" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/nodes\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"
Oct 25 19:11:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:42.007378    4021 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn?timeout=10s\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" interval="7s"
Oct 25 19:11:42 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:42.761721    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:43.096699    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:43.096730    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:43.217987    4021 kubelet_node_status.go:376] "Setting node annotation to enable volume controller attach/detach"
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:43.218532    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientMemory"
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:43.218623    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasNoDiskPressure"
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:43.218697    4021 kubelet_node_status.go:696] "Recording event message for node" node="nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" event="NodeHasSufficientPID"
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:43.344532    4021 remote_runtime.go:319] "CreateContainer in sandbox from runtime service failed" err=<
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:         rpc error: code = Unknown desc = container create failed: time="2023-10-25T19:11:43Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-0035dd07a203970271ff37aa91aa4ca54fd32fc695923f3dc00f9e7fb3f453ea.scope/cpuset.cpus: permission denied"
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]:  > podSandboxID="e8851e2bcfe2f0a129baf735bd07696b15aa0143e3003543f2de561928f5cc3b"
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:43.344634    4021 kuberuntime_manager.go:1256] init container &Container{Name:render-config-keepalived,Image:registry.build02.ci.openshift.org/ci-op-nndncs4w/stable@sha256:ebfae1ad38c560436f4ed874fd8969329c946a57c3ccd1aed6750a1597cd39e8,Command:[runtimecfg render /etc/kubernetes/kubeconfig --api-vips 10.0.0.5 --ingress-vips 10.0.0.7 /config --out-dir /etc/keepalived],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kubeconfig,ReadOnly:false,MountPath:/etc/kubernetes,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:script-dir,ReadOnly:false,MountPath:/config,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:conf-dir,ReadOnly:false,MountPath:/etc/keepalived,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},VolumeMount{Name:nodeip-configuration,ReadOnly:false,MountPath:/run/nodeip-configuration,SubPath:,MountPropagation:*HostToContainer,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn_openshift-openstack-infra(ef062c82f679a1ef42be9dd35e115e2f): CreateContainerError: container create failed: time="2023-10-25T19:11:43Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \"0-7\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-0035dd07a203970271ff37aa91aa4ca54fd32fc695923f3dc00f9e7fb3f453ea.scope/cpuset.cpus: permission denied"
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:43.344689    4021 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"render-config-keepalived\" with CreateContainerError: \"container create failed: time=\\\"2023-10-25T19:11:43Z\\\" level=error msg=\\\"runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write \\\\\\\"0-7\\\\\\\": write /sys/fs/cgroup/cpuset/system.slice/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef062c82f679a1ef42be9dd35e115e2f.slice/crio-0035dd07a203970271ff37aa91aa4ca54fd32fc695923f3dc00f9e7fb3f453ea.scope/cpuset.cpus: permission denied\\\"\\n\"" pod="openshift-openstack-infra/keepalived-nndncs4w-b3ed8-rw8t5-worker-0-bbpcn" podUID="ef062c82f679a1ef42be9dd35e115e2f"
Oct 25 19:11:43 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:43.764384    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:44.148528    4021 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn\" not found"
Oct 25 19:11:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:44.765349    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:44.892283    4021 webhook.go:223] Failed to make webhook authorizer request: Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:44 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:44.892392    4021 server.go:325] "Authorization error" err="Post \"https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host" user="system:serviceaccount:openshift-monitoring:prometheus-k8s" verb="get" resource="nodes" subresource="metrics"
Oct 25 19:11:45 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:45.765513    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:46 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: I1025 19:11:46.765841    4021 csi_plugin.go:913] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/nndncs4w-b3ed8-rw8t5-worker-0-bbpcn": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host
Oct 25 19:11:47 nndncs4w-b3ed8-rw8t5-worker-0-bbpcn kubenswrapper[4021]: E1025 19:11:47.292645    4021 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn.17916ebfb1d0cf98", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", UID:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}, FirstTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), LastTimestamp:time.Date(2023, time.October, 25, 18, 58, 24, 96448408, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"nndncs4w-b3ed8-rw8t5-worker-0-bbpcn"}': 'Post "https://api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com:6443/api/v1/namespaces/default/events": dial tcp: lookup api-int.nndncs4w-b3ed8.shiftstack.devcluster.openshift.com on 199.204.44.24:53: no such host'(may retry after sleeping)