2026-02-09T13:24:07.10109988Z	INFO	setup	Go Version: go1.25.3 (Red Hat 1.25.3-1.el9_7) X:strictfipsruntime
2026-02-09T13:24:07.101594121Z	INFO	setup	Go OS/Arch: linux/amd64
2026-02-09T13:24:07.101599793Z	INFO	setup	Operator Version: 333f49c
2026-02-09T13:24:07.101602325Z	INFO	setup	Git Commit: 333f49ce59a566b59f89df5d44789e6d55871a25
2026-02-09T13:24:07.101611335Z	INFO	setup	Build Date: 2026-02-02T11:55:43+00:00
2026-02-09T13:24:07.10162065Z	INFO	setup	HTTP/2 for metrics and webhook server disabled
2026-02-09T13:24:07.146492787Z	INFO	setup	supported control plane topology	{"topology": "HighlyAvailable"}
2026-02-09T13:24:07.146548908Z	INFO	setup	Cluster capabilities	{"IsOnOpenshift": true, "HasMachineAPI": true}
2026-02-09T13:24:07.146958088Z	INFO	controller-runtime.builder	Registering a validating webhook	{"GVK": "remediation.medik8s.io/v1alpha1, Kind=NodeHealthCheck", "path": "/validate-remediation-medik8s-io-v1alpha1-nodehealthcheck"}
2026-02-09T13:24:07.1470862Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/validate-remediation-medik8s-io-v1alpha1-nodehealthcheck"}
2026-02-09T13:24:07.147136612Z	INFO	setup	starting manager
2026-02-09T13:24:07.147192513Z	INFO	controller-runtime.metrics	Starting metrics server
2026-02-09T13:24:07.147311837Z	INFO	controller-runtime.metrics	Serving metrics server	{"bindAddress": "127.0.0.1:8080", "secure": false}
2026-02-09T13:24:07.147392305Z	INFO	starting server	{"name": "health probe", "addr": "[::]:8081"}
2026-02-09T13:24:07.147416706Z	INFO	controller-runtime.webhook	Starting webhook server
2026-02-09T13:24:07.147666615Z	INFO	controller-runtime.certwatcher	Updated current TLS certificate
2026-02-09T13:24:07.147774071Z	INFO	controller-runtime.webhook	Serving webhook server	{"host": "", "port": 9443}
2026-02-09T13:24:07.147881786Z	INFO	controller-runtime.certwatcher	Starting certificate poll+watcher	{"interval": "10s"}
I0209 13:24:07.248196       1 leaderelection.go:257] attempting to acquire leader lease openshift-workload-availability/e1f13584.medik8s.io...
I0209 13:24:54.134986       1 leaderelection.go:271] successfully acquired lease openshift-workload-availability/e1f13584.medik8s.io
2026-02-09T13:24:54.135040537Z	DEBUG	events	node-healthcheck-controller-manager-5df98d9f6d-2n9zk_f81e4774-bd3f-48fb-8179-457248d286ce became leader	{"type": "Normal", "object": {"kind":"Lease","namespace":"openshift-workload-availability","name":"e1f13584.medik8s.io","uid":"423bbe22-ee47-467c-b4ce-cd56cc584014","apiVersion":"coordination.k8s.io/v1","resourceVersion":"101196"}, "reason": "LeaderElection"}
2026-02-09T13:24:54.135716224Z	INFO	Starting EventSource	{"controller": "nodehealthcheck", "controllerGroup": "remediation.medik8s.io", "controllerKind": "NodeHealthCheck", "source": "channel source: 0xc0005f6690"}
2026-02-09T13:24:54.135807185Z	INFO	Starting EventSource	{"controller": "nodehealthcheck", "controllerGroup": "remediation.medik8s.io", "controllerKind": "NodeHealthCheck", "source": "kind source: *v1alpha1.NodeHealthCheck"}
2026-02-09T13:24:54.135830954Z	INFO	Starting EventSource	{"controller": "nodehealthcheck", "controllerGroup": "remediation.medik8s.io", "controllerKind": "NodeHealthCheck", "source": "kind source: *v1.Node"}
2026-02-09T13:24:54.13586532Z	INFO	Starting EventSource	{"controller": "machinehealthcheck", "controllerGroup": "machine.openshift.io", "controllerKind": "MachineHealthCheck", "source": "channel source: 0xc0005f68c0"}
2026-02-09T13:24:54.135907484Z	INFO	Starting EventSource	{"controller": "machinehealthcheck", "controllerGroup": "machine.openshift.io", "controllerKind": "MachineHealthCheck", "source": "kind source: *v1beta1.MachineHealthCheck"}
2026-02-09T13:24:54.135924662Z	INFO	Starting EventSource	{"controller": "machinehealthcheck", "controllerGroup": "machine.openshift.io", "controllerKind": "MachineHealthCheck", "source": "kind source: *v1.Node"}
2026-02-09T13:24:54.13591318Z	INFO	Starting EventSource	{"controller": "machinehealthcheck", "controllerGroup": "machine.openshift.io", "controllerKind": "MachineHealthCheck", "source": "kind source: *v1beta1.Machine"}
I0209 13:24:54.137540       1 shared_informer.go:350] "Waiting for caches to sync" controller="feature gate accessor"
2026-02-09T13:24:54.236954434Z	INFO	MHCChecker	found custom MHC, will disable NHC
2026-02-09T13:24:54.23699869Z	INFO	MHCChecker	MHC Checker status changed, notifying NHC controller
2026-02-09T13:24:54.237005676Z	INFO	Starting Controller	{"controller": "machinehealthcheck", "controllerGroup": "machine.openshift.io", "controllerKind": "MachineHealthCheck"}
2026-02-09T13:24:54.237023496Z	INFO	Starting workers	{"controller": "machinehealthcheck", "controllerGroup": "machine.openshift.io", "controllerKind": "MachineHealthCheck", "worker count": 1}
2026-02-09T13:24:54.236956384Z	INFO	Starting Controller	{"controller": "nodehealthcheck", "controllerGroup": "remediation.medik8s.io", "controllerKind": "NodeHealthCheck"}
2026-02-09T13:24:54.23704707Z	INFO	Starting workers	{"controller": "nodehealthcheck", "controllerGroup": "remediation.medik8s.io", "controllerKind": "NodeHealthCheck", "worker count": 1}
2026-02-09T13:24:54.237096743Z	INFO	adding all NHCs to reconcile queue for handling MHC event
I0209 13:24:54.238142       1 shared_informer.go:357] "Caches are synced" controller="feature gate accessor"
I0209 13:24:54.238179       1 simple_featuregate_reader.go:171] Starting feature-gate-detector
2026-02-09T13:24:54.238270826Z	INFO	FeatureGateAccessor	FeatureGates initialized
I0209 13:24:54.238405       1 recorder_logging.go:49] &Event{ObjectMeta:{dummy.189296e071edcc5f.9d048c1a  dummy    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:dummy,Name:dummy,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:FeatureGatesInitialized,Message:FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "NewOLM", "PinnedImages", "PreconfiguredUDNAddresses", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}},Source:EventSource{Component:,Host:,},FirstTimestamp:2026-02-09 13:24:54.238276703 +0000 UTC m=+47.358441570,LastTimestamp:2026-02-09 13:24:54.238276703 +0000 UTC m=+47.358441570,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:,ReportingInstance:,}
2026-02-09T13:24:54.559766684Z	INFO	console-plugin	successfully created / updated console plugin resources
2026-02-09T13:29:03.867104717Z	INFO	MHCChecker	found termination handler MHC, will ignore Nodes with Terminating condition
2026-02-09T13:29:03.867136913Z	INFO	MHCChecker	MHC Checker status changed, notifying NHC controller
2026-02-09T13:29:03.867185039Z	INFO	adding all NHCs to reconcile queue for handling MHC event
2026-02-09T15:05:21.211326462Z	INFO	nodehealthcheck-resource	validate create	{"name": "nhc-worker-test"}
2026-02-09T15:05:21.215297391Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:05:21.220605499Z	INFO	controllers.NodeHealthCheck	disabling NHC	{"NodeHealthCheck name": "nhc-worker-test", "reason": "RemediationTemplateNotFound", "message": "Remediation template not found: \"failed to get external remediation template openshift-workload-availability/selfnoderemediationtemplate-sample: selfnoderemediationtemplates.self-node-remediation.medik8s.io \\\"selfnoderemediationtemplate-sample\\\" not found\""}
2026-02-09T15:05:21.220829447Z	DEBUG	events	[remediation] Disabling NHC. Reason: RemediationTemplateNotFound, Message: Remediation template not found: "failed to get external remediation template openshift-workload-availability/selfnoderemediationtemplate-sample: selfnoderemediationtemplates.self-node-remediation.medik8s.io \"selfnoderemediationtemplate-sample\" not found"	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"134338"}, "reason": "Disabled"}
2026-02-09T15:05:21.221134679Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":0,"healthyNodes":0,"conditions":[{"type":"Disabled","status":"True","lastTransitionTime":"2026-02-09T15:05:21Z","reason":"RemediationTemplateNotFound","message":"Remediation template not found: \"failed to get external remediation template openshift-workload-availability/selfnoderemediationtemplate-sample: selfnoderemediationtemplates.self-node-remediation.medik8s.io \\\"selfnoderemediationtemplate-sample\\\" not found\""}],"phase":"Disabled","reason":"NHC is disabled: RemediationTemplateNotFound: Remediation template not found: \"failed to get external remediation template openshift-workload-availability/selfnoderemediationtemplate-sample: selfnoderemediationtemplates.self-node-remediation.medik8s.io \\\"selfnoderemediationtemplate-sample\\\" not found\""}, "patch": "{\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2026-02-09T15:05:21Z\",\"message\":\"Remediation template not found: \\\"failed to get external remediation template openshift-workload-availability/selfnoderemediationtemplate-sample: selfnoderemediationtemplates.self-node-remediation.medik8s.io \\\\\\\"selfnoderemediationtemplate-sample\\\\\\\" not found\\\"\",\"reason\":\"RemediationTemplateNotFound\",\"status\":\"True\",\"type\":\"Disabled\"}],\"healthyNodes\":0,\"observedNodes\":0,\"phase\":\"Disabled\",\"reason\":\"NHC is disabled: RemediationTemplateNotFound: Remediation template not found: \\\"failed to get external remediation template openshift-workload-availability/selfnoderemediationtemplate-sample: selfnoderemediationtemplates.self-node-remediation.medik8s.io \\\\\\\"selfnoderemediationtemplate-sample\\\\\\\" not found\\\"\"}}"}
2026-02-09T15:05:21.428191451Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "15s"}
2026-02-09T15:05:36.428624289Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:05:36.433800467Z	INFO	controllers.NodeHealthCheck	enabling NHC, valid config, no conflicting MHC configured in the cluster	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:05:36.433890451Z	INFO	Starting EventSource	{"controller": "nodehealthcheck", "controllerGroup": "remediation.medik8s.io", "controllerKind": "NodeHealthCheck", "source": "kind source: *unstructured.Unstructured"}
2026-02-09T15:05:36.433915052Z	INFO	controllers.NodeHealthCheck.WatchManager	added watch for remediation template CRs	{"kind": "SelfNodeRemediationTemplate"}
2026-02-09T15:05:36.433925272Z	INFO	Starting EventSource	{"controller": "nodehealthcheck", "controllerGroup": "remediation.medik8s.io", "controllerKind": "NodeHealthCheck", "source": "kind source: *unstructured.Unstructured"}
2026-02-09T15:05:36.433930065Z	INFO	controllers.NodeHealthCheck.WatchManager	added watch for remediation CRs	{"kind": "SelfNodeRemediation"}
2026-02-09T15:05:36.433956495Z	DEBUG	events	[remediation] No issues found, NodeHealthCheck is enabled.	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"134339"}, "reason": "Enabled"}
2026-02-09T15:05:36.440559314Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:05:36Z" level=info msg="invalidating lease"
time="2026-02-09T15:05:36Z" level=info msg="getting lease"
2026-02-09T15:05:36.54363063Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
time="2026-02-09T15:05:36Z" level=info msg="invalidating lease"
time="2026-02-09T15:05:36Z" level=info msg="getting lease"
2026-02-09T15:05:36.546931576Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":2,"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."}],"phase":"Enabled","reason":"NHC is enabled, no ongoing remediation","lastUpdateTime":"2026-02-09T15:05:21Z"}, "patch": "{\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2026-02-09T15:05:36Z\",\"message\":\"No issues found, NodeHealthCheck is enabled.\",\"reason\":\"NodeHealthCheckEnabled\",\"status\":\"False\",\"type\":\"Disabled\"}],\"healthyNodes\":2,\"observedNodes\":2,\"phase\":\"Enabled\",\"reason\":\"NHC is enabled, no ongoing remediation\"}}"}
2026-02-09T15:05:36.75422997Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "0s"}
2026-02-09T15:29:28.414342136Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-1", "NHC": "nhc-worker-test"}
2026-02-09T15:29:28.414511173Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-1", "NHC": "nhc-worker-test"}
2026-02-09T15:29:28.414665243Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:29:28.427829147Z	INFO	controllers.NodeHealthCheck	Node is going to match unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown", "duration left": "29.572175145s"}
2026-02-09T15:29:28.448972755Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
time="2026-02-09T15:29:28Z" level=info msg="invalidating lease"
time="2026-02-09T15:29:28Z" level=info msg="getting lease"
2026-02-09T15:29:28.548687028Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":1,"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."}],"phase":"Enabled","reason":"NHC is enabled, no ongoing remediation","lastUpdateTime":"2026-02-09T15:05:36Z"}, "patch": "{\"status\":{\"healthyNodes\":1}}"}
2026-02-09T15:29:28.757530565Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "30.572175145s"}
2026-02-09T15:29:59.330458909Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:29:59.336887387Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:29:59.33707235Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-1", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"141465"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:29:59.342036581Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
time="2026-02-09T15:29:59Z" level=info msg="invalidating lease"
time="2026-02-09T15:29:59Z" level=info msg="getting lease"
2026-02-09T15:29:59.344154986Z	INFO	controllers.NodeHealthCheck	handling unhealthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T15:29:59.348878479Z	INFO	controllers.NodeHealthCheck.resource manager	Attempting to obtain Node Lease	{"Node name": "worker-0-1"}
time="2026-02-09T15:29:59Z" level=info msg="request lease"
time="2026-02-09T15:29:59Z" level=info msg="getting lease"
time="2026-02-09T15:29:59Z" level=info msg="create lease"
2026-02-09T15:29:59.352908949Z	INFO	controllers.NodeHealthCheck.resource manager	Creating a remediation CR	{"CR name": "", "CR kind": "SelfNodeRemediation", "namespace": "openshift-workload-availability"}
2026-02-09T15:29:59.363681281Z	DEBUG	events	[remediation] Created remediation object for node worker-0-1	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"141465"}, "reason": "RemediationCreated"}
2026-02-09T15:29:59.366559443Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":1,"unhealthyNodes":[{"name":"worker-0-1","remediations":[{"resource":{"kind":"SelfNodeRemediation","namespace":"openshift-workload-availability","name":"worker-0-1-m25mj","uid":"4b484684-5811-4fa7-a1af-1d3ac6f05c98","apiVersion":"self-node-remediation.medik8s.io/v1alpha1"},"started":"2026-02-09T15:29:59Z","templateName":"selfnoderemediationtemplate-sample"}]}],"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."}],"phase":"Remediating","reason":"NHC is remediating 1 nodes","lastUpdateTime":"2026-02-09T15:29:28Z"}, "patch": "{\"status\":{\"phase\":\"Remediating\",\"reason\":\"NHC is remediating 1 nodes\",\"unhealthyNodes\":[{\"name\":\"worker-0-1\",\"remediations\":[{\"resource\":{\"apiVersion\":\"self-node-remediation.medik8s.io/v1alpha1\",\"kind\":\"SelfNodeRemediation\",\"name\":\"worker-0-1-m25mj\",\"namespace\":\"openshift-workload-availability\",\"uid\":\"4b484684-5811-4fa7-a1af-1d3ac6f05c98\"},\"started\":\"2026-02-09T15:29:59Z\",\"templateName\":\"selfnoderemediationtemplate-sample\"}]}]}}"}
2026-02-09T15:29:59.574203965Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "10m0s"}
2026-02-09T15:29:59.578415877Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:29:59.578513137Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:29:59.578540272Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:29:59.578551937Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:29:59.578607307Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:29:59.584192101Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:29:59.584240053Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:29:59.584257836Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:29:59.584269187Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:29:59.585238571Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:29:59.590942227Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
time="2026-02-09T15:29:59Z" level=info msg="invalidating lease"
time="2026-02-09T15:29:59Z" level=info msg="getting lease"
2026-02-09T15:29:59.59407095Z	INFO	controllers.NodeHealthCheck	handling unhealthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T15:29:59.610004823Z	INFO	controllers.NodeHealthCheck.resource manager	external remediation CR already exists	{"CR name": "worker-0-1-m25mj", "kind": "SelfNodeRemediation", "namespace": "openshift-workload-availability"}
time="2026-02-09T15:29:59Z" level=info msg="getting lease"
2026-02-09T15:29:59.610182736Z	INFO	controllers.NodeHealthCheck.nhc lease manager	managing lease - about to try to acquire/extended the lease	{"NodeHealthCheck name": "nhc-worker-test", "lease name": "node-worker-0-1", "NHC is lease owner": true, "lease expiration time": "10m0s"}
2026-02-09T15:29:59.639205298Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:29:59.639260477Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:29:59.639276811Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:29:59.639282907Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:29:59.640854505Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "10m0s"}
2026-02-09T15:29:59.640971028Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:29:59.659057005Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:29:59.70980223Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:29:59Z"}
2026-02-09T15:29:59.709893038Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
time="2026-02-09T15:29:59Z" level=info msg="invalidating lease"
time="2026-02-09T15:29:59Z" level=info msg="getting lease"
2026-02-09T15:29:59.810603224Z	INFO	controllers.NodeHealthCheck	handling unhealthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T15:29:59.909502867Z	INFO	controllers.NodeHealthCheck.resource manager	external remediation CR already exists	{"CR name": "worker-0-1-m25mj", "kind": "SelfNodeRemediation", "namespace": "openshift-workload-availability"}
time="2026-02-09T15:29:59Z" level=info msg="getting lease"
2026-02-09T15:29:59.90961775Z	INFO	controllers.NodeHealthCheck.nhc lease manager	managing lease - about to try to acquire/extended the lease	{"NodeHealthCheck name": "nhc-worker-test", "lease name": "node-worker-0-1", "NHC is lease owner": true, "lease expiration time": "10m0s"}
2026-02-09T15:30:00.010347643Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "10m0s"}
2026-02-09T15:30:00.64862679Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:30:00.648688303Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:30:00.648710711Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:30:00.648718474Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:30:00.648834361Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:30:00.666283485Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:30:00.673527738Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:29:59Z"}
2026-02-09T15:30:00.673571188Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
time="2026-02-09T15:30:00Z" level=info msg="invalidating lease"
time="2026-02-09T15:30:00Z" level=info msg="getting lease"
2026-02-09T15:30:00.6770821Z	INFO	controllers.NodeHealthCheck	handling unhealthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T15:30:00.774792374Z	INFO	controllers.NodeHealthCheck.resource manager	external remediation CR already exists	{"CR name": "worker-0-1-m25mj", "kind": "SelfNodeRemediation", "namespace": "openshift-workload-availability"}
time="2026-02-09T15:30:00Z" level=info msg="getting lease"
2026-02-09T15:30:00.77492227Z	INFO	controllers.NodeHealthCheck.nhc lease manager	managing lease - about to try to acquire/extended the lease	{"NodeHealthCheck name": "nhc-worker-test", "lease name": "node-worker-0-1", "NHC is lease owner": true, "lease expiration time": "10m0s"}
2026-02-09T15:30:00.778600802Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "10m0s"}
2026-02-09T15:31:14.092158873Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-2", "NHC": "nhc-worker-test"}
2026-02-09T15:31:14.092257543Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-2", "NHC": "nhc-worker-test"}
2026-02-09T15:31:14.09232882Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:14.105718919Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:31:14.105804257Z	INFO	controllers.NodeHealthCheck	Node is going to match unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown", "duration left": "29.894197155s"}
2026-02-09T15:31:14.128540243Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:29:59Z"}
2026-02-09T15:31:14.128607074Z	INFO	controllers.NodeHealthCheck	Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:14.12864481Z	INFO	controllers.NodeHealthCheck	Storm recovery mode activated	{"nhc": "nhc-worker-test"}
2026-02-09T15:31:14.128785604Z	DEBUG	events	[remediation] Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"141842"}, "reason": "RemediationSkipped"}
2026-02-09T15:31:14.128827118Z	DEBUG	events	[remediation] Storm recovery mode activated - delaying remediation until threshold is reached	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"141842"}, "reason": "StormRecoveryStarted"}
2026-02-09T15:31:14.129308778Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":0,"unhealthyNodes":[{"name":"worker-0-1","remediations":[{"resource":{"kind":"SelfNodeRemediation","namespace":"openshift-workload-availability","name":"worker-0-1-m25mj","uid":"4b484684-5811-4fa7-a1af-1d3ac6f05c98","apiVersion":"self-node-remediation.medik8s.io/v1alpha1"},"started":"2026-02-09T15:29:59Z","templateName":"selfnoderemediationtemplate-sample"}]}],"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"False","lastTransitionTime":"2026-02-09T15:31:14Z","reason":"HealthyNodeThresholdChange","message":"Cooldown cleared - storm threshold triggered"},{"type":"StormActive","status":"True","lastTransitionTime":"2026-02-09T15:31:14Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is activated - preventing any new remediation until the storm is over and cooldown duration expired"}],"phase":"Remediating","reason":"NHC is remediating 1 nodes","lastUpdateTime":"2026-02-09T15:29:59Z"}, "patch": "{\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2026-02-09T15:05:36Z\",\"message\":\"No issues found, NodeHealthCheck is enabled.\",\"reason\":\"NodeHealthCheckEnabled\",\"status\":\"False\",\"type\":\"Disabled\"},{\"lastTransitionTime\":\"2026-02-09T15:31:14Z\",\"message\":\"Cooldown cleared - storm threshold triggered\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"False\",\"type\":\"StormCooldownActive\"},{\"lastTransitionTime\":\"2026-02-09T15:31:14Z\",\"message\":\"Storm mode is activated - preventing any new remediation until the storm is over and cooldown duration expired\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"True\",\"type\":\"StormActive\"}],\"healthyNodes\":0}}"}
2026-02-09T15:31:14.343947214Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "30.894197155s"}
2026-02-09T15:31:45.238686831Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:45.244078228Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:31:45.244210506Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:31:45.244274633Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"142565"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:31:45.250307739Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:29:59Z"}
2026-02-09T15:31:45.250416501Z	INFO	controllers.NodeHealthCheck	Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:45.250768528Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":0,"unhealthyNodes":[{"name":"worker-0-1","remediations":[{"resource":{"kind":"SelfNodeRemediation","namespace":"openshift-workload-availability","name":"worker-0-1-m25mj","uid":"4b484684-5811-4fa7-a1af-1d3ac6f05c98","apiVersion":"self-node-remediation.medik8s.io/v1alpha1"},"started":"2026-02-09T15:29:59Z","templateName":"selfnoderemediationtemplate-sample"}]},{"name":"worker-0-2"}],"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"False","lastTransitionTime":"2026-02-09T15:31:14Z","reason":"HealthyNodeThresholdChange","message":"Cooldown cleared - storm threshold triggered"},{"type":"StormActive","status":"True","lastTransitionTime":"2026-02-09T15:31:14Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is activated - preventing any new remediation until the storm is over and cooldown duration expired"}],"phase":"Remediating","reason":"NHC is remediating 2 nodes","lastUpdateTime":"2026-02-09T15:31:14Z"}, "patch": "{\"status\":{\"reason\":\"NHC is remediating 2 nodes\",\"unhealthyNodes\":[{\"name\":\"worker-0-1\",\"remediations\":[{\"resource\":{\"apiVersion\":\"self-node-remediation.medik8s.io/v1alpha1\",\"kind\":\"SelfNodeRemediation\",\"name\":\"worker-0-1-m25mj\",\"namespace\":\"openshift-workload-availability\",\"uid\":\"4b484684-5811-4fa7-a1af-1d3ac6f05c98\"},\"started\":\"2026-02-09T15:29:59Z\",\"templateName\":\"selfnoderemediationtemplate-sample\"}]},{\"name\":\"worker-0-2\"}]}}"}
2026-02-09T15:31:45.251181802Z	DEBUG	events	[remediation] Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"142565"}, "reason": "RemediationSkipped"}
2026-02-09T15:31:45.458233987Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "0s"}
2026-02-09T15:31:51.70658248Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-1", "NHC": "nhc-worker-test"}
2026-02-09T15:31:51.706676375Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-1", "NHC": "nhc-worker-test"}
2026-02-09T15:31:51.706769516Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:51.726338719Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:31:51.726629618Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143222"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:31:51.73762383Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:29:59Z"}
2026-02-09T15:31:51.737671901Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T15:31:51.83832531Z	INFO	controllers.NodeHealthCheck.resource manager	setting a delay for node getting healthy	{"node name": "worker-0-1", "delay in seconds": 5}
2026-02-09T15:31:51.846822685Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:31:51.846885186Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:31:51.846898183Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:31:51.846903754Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:31:51.84695016Z	INFO	controllers.NodeHealthCheck	Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:51.847082745Z	DEBUG	events	[remediation] Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143222"}, "reason": "RemediationSkipped"}
2026-02-09T15:31:51.847364232Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":0,"unhealthyNodes":[{"name":"worker-0-1","remediations":[{"resource":{"kind":"SelfNodeRemediation","namespace":"openshift-workload-availability","name":"worker-0-1-m25mj","uid":"4b484684-5811-4fa7-a1af-1d3ac6f05c98","apiVersion":"self-node-remediation.medik8s.io/v1alpha1"},"started":"2026-02-09T15:29:59Z","templateName":"selfnoderemediationtemplate-sample"}],"conditionsHealthyTimestamp":"2026-02-09T15:31:51Z","healthyDelayed":true},{"name":"worker-0-2"}],"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"False","lastTransitionTime":"2026-02-09T15:31:14Z","reason":"HealthyNodeThresholdChange","message":"Cooldown cleared - storm threshold triggered"},{"type":"StormActive","status":"True","lastTransitionTime":"2026-02-09T15:31:14Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is activated - preventing any new remediation until the storm is over and cooldown duration expired"}],"phase":"Remediating","reason":"NHC is remediating 2 nodes","lastUpdateTime":"2026-02-09T15:31:45Z"}, "patch": "{\"status\":{\"unhealthyNodes\":[{\"conditionsHealthyTimestamp\":\"2026-02-09T15:31:51Z\",\"healthyDelayed\":true,\"name\":\"worker-0-1\",\"remediations\":[{\"resource\":{\"apiVersion\":\"self-node-remediation.medik8s.io/v1alpha1\",\"kind\":\"SelfNodeRemediation\",\"name\":\"worker-0-1-m25mj\",\"namespace\":\"openshift-workload-availability\",\"uid\":\"4b484684-5811-4fa7-a1af-1d3ac6f05c98\"},\"started\":\"2026-02-09T15:29:59Z\",\"templateName\":\"selfnoderemediationtemplate-sample\"}]},{\"name\":\"worker-0-2\"}]}}"}
2026-02-09T15:31:52.058336959Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "6s"}
2026-02-09T15:31:52.058439768Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:52.06466324Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:31:52.064797437Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:31:52.075421741Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:29:59Z"}
2026-02-09T15:31:52.075463796Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T15:31:52.078052947Z	INFO	controllers.NodeHealthCheck.resource manager	delaying node getting healthy	{"node name": "worker-0-1", "remaining time in seconds": 3.921948619}
2026-02-09T15:31:52.078080424Z	INFO	controllers.NodeHealthCheck	Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:52.078149187Z	DEBUG	events	[remediation] Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "RemediationSkipped"}
2026-02-09T15:31:52.078357667Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "4.921948619s"}
2026-02-09T15:31:57.000809857Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:57.007543319Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:31:57.007651048Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:31:57.013372475Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:29:59Z"}
2026-02-09T15:31:57.0133957Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T15:31:57.016230826Z	INFO	controllers.NodeHealthCheck.resource manager	delaying for node getting healthy is done, about to remove the remediation CR	{"node name": "worker-0-1"}
2026-02-09T15:31:57.024259231Z	INFO	controllers.NodeHealthCheck.resource manager	deleted remediation CR	{"name": "worker-0-1-m25mj"}
2026-02-09T15:31:57.024287335Z	DEBUG	events	[remediation] Deleted remediation CR of kind SelfNodeRemediation with name worker-0-1-m25mj	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "RemediationRemoved"}
2026-02-09T15:31:57.024305022Z	INFO	controllers.NodeHealthCheck	Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:57.024414119Z	DEBUG	events	[remediation] Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "RemediationSkipped"}
2026-02-09T15:31:57.024445708Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:31:57.024466989Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:31:57.024479071Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:31:57.024485074Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:31:57.024689677Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "11s"}
2026-02-09T15:31:57.024794423Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:57.030596296Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:31:57.030730552Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:31:57.114825952Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T15:31:57.215927008Z	INFO	controllers.NodeHealthCheck.resource manager	delaying for node getting healthy is done, about to remove the remediation CR	{"node name": "worker-0-1"}
2026-02-09T15:31:57.218699242Z	INFO	controllers.NodeHealthCheck	Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:31:57.218887785Z	DEBUG	events	[remediation] Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "RemediationSkipped"}
2026-02-09T15:31:57.21907799Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "17s"}
2026-02-09T15:32:01.013863158Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:32:01.013921194Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:32:01.013945817Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:32:01.01395469Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:32:01.014014995Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:32:01.018991144Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:32:01.01916234Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:32:01.04399261Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T15:32:01.140246088Z	INFO	controllers.NodeHealthCheck.resource manager	delaying for node getting healthy is done, about to remove the remediation CR	{"node name": "worker-0-1"}
2026-02-09T15:32:01.143223106Z	INFO	controllers.NodeHealthCheck	Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:32:01.143384293Z	DEBUG	events	[remediation] Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "RemediationSkipped"}
2026-02-09T15:32:01.143548906Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "17s"}
2026-02-09T15:32:01.154845592Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:32:01.154887162Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:32:01.154908274Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:32:01.154917077Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:32:01.154969431Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:32:01.16483738Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:32:01.16503979Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:32:01.240198281Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T15:32:01.242878297Z	INFO	controllers.NodeHealthCheck.resource manager	delaying for node getting healthy is done, about to remove the remediation CR	{"node name": "worker-0-1"}
2026-02-09T15:32:01.246118747Z	INFO	controllers.NodeHealthCheck	Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:32:01.246245803Z	DEBUG	events	[remediation] Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "RemediationSkipped"}
2026-02-09T15:32:01.246441998Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "17s"}
2026-02-09T15:32:02.20530831Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:32:02.205381676Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-1-m25mj", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:32:02.205471658Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:32:02.226340397Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:32:02.226544482Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:32:02.249137951Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:32:02Z" level=info msg="invalidating lease"
time="2026-02-09T15:32:02Z" level=info msg="getting lease"
2026-02-09T15:32:02.354334213Z	INFO	controllers.NodeHealthCheck	The cluster regained health after the storm, a cooldown period now begins as a safety measure before normal operations resume.	{"nhc": "nhc-worker-test"}
2026-02-09T15:32:02.354391913Z	INFO	controllers.NodeHealthCheck	Storm recovery active: skipping creation of new remediations
2026-02-09T15:32:02.354551416Z	DEBUG	events	[remediation] Storm recovery mode will exit after cooldown delay	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "StormRecoveryCooldownStarted"}
2026-02-09T15:32:02.354605899Z	DEBUG	events	[remediation] Storm recovery active: skipping creation of new remediations	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143278"}, "reason": "RemediationSkipped"}
2026-02-09T15:32:02.35469936Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":1,"unhealthyNodes":[{"name":"worker-0-2"}],"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"True","lastTransitionTime":"2026-02-09T15:32:02Z","reason":"HealthyNodeThresholdChange","message":"Storm cooldown delay started - waiting before resuming normal remediation"},{"type":"StormActive","status":"True","lastTransitionTime":"2026-02-09T15:31:14Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is activated - preventing any new remediation until the storm is over and cooldown duration expired"}],"phase":"Enabled","reason":"NHC is enabled, no ongoing remediation","lastUpdateTime":"2026-02-09T15:31:51Z"}, "patch": "{\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2026-02-09T15:05:36Z\",\"message\":\"No issues found, NodeHealthCheck is enabled.\",\"reason\":\"NodeHealthCheckEnabled\",\"status\":\"False\",\"type\":\"Disabled\"},{\"lastTransitionTime\":\"2026-02-09T15:32:02Z\",\"message\":\"Storm cooldown delay started - waiting before resuming normal remediation\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"True\",\"type\":\"StormCooldownActive\"},{\"lastTransitionTime\":\"2026-02-09T15:31:14Z\",\"message\":\"Storm mode is activated - preventing any new remediation until the storm is over and cooldown duration expired\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"True\",\"type\":\"StormActive\"}],\"healthyNodes\":1,\"phase\":\"Enabled\",\"reason\":\"NHC is enabled, no ongoing remediation\",\"unhealthyNodes\":[{\"name\":\"worker-0-2\"}]}}"}
2026-02-09T15:32:02.56525535Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "1m1s"}
2026-02-09T15:32:08.025191848Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:32:08.031531519Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:32:08.031687558Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143745"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:32:08.037579658Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:32:08Z" level=info msg="invalidating lease"
time="2026-02-09T15:32:08Z" level=info msg="getting lease"
2026-02-09T15:32:08.040050352Z	INFO	controllers.NodeHealthCheck	Storm recovery active: skipping creation of new remediations
2026-02-09T15:32:08.040168201Z	DEBUG	events	[remediation] Storm recovery active: skipping creation of new remediations	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143745"}, "reason": "RemediationSkipped"}
2026-02-09T15:32:08.040307544Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "54.95995101s"}
2026-02-09T15:33:03.001228001Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:33:03.007159779Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:33:03.007372413Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143745"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:33:03.013471873Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:33:03Z" level=info msg="invalidating lease"
time="2026-02-09T15:33:03Z" level=info msg="getting lease"
2026-02-09T15:33:03.015297722Z	INFO	controllers.NodeHealthCheck	Storm recovery mode deactivated	{"nhc": "nhc-worker-test"}
2026-02-09T15:33:03.015326361Z	INFO	controllers.NodeHealthCheck	handling unhealthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:33:03.015408247Z	DEBUG	events	[remediation] Storm recovery mode deactivated - normal remediation resumed	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143745"}, "reason": "StormRecoveryEnded"}
2026-02-09T15:33:03.114907054Z	INFO	controllers.NodeHealthCheck.resource manager	Attempting to obtain Node Lease	{"Node name": "worker-0-2"}
time="2026-02-09T15:33:03Z" level=info msg="request lease"
time="2026-02-09T15:33:03Z" level=info msg="getting lease"
time="2026-02-09T15:33:03Z" level=info msg="create lease"
2026-02-09T15:33:03.119297491Z	INFO	controllers.NodeHealthCheck.resource manager	Creating a remediation CR	{"CR name": "", "CR kind": "SelfNodeRemediation", "namespace": "openshift-workload-availability"}
2026-02-09T15:33:03.12628694Z	DEBUG	events	[remediation] Created remediation object for node worker-0-2	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"143745"}, "reason": "RemediationCreated"}
2026-02-09T15:33:03.130866985Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":1,"unhealthyNodes":[{"name":"worker-0-2","remediations":[{"resource":{"kind":"SelfNodeRemediation","namespace":"openshift-workload-availability","name":"worker-0-2-jg86c","uid":"77ad3a6d-39ab-4495-a45a-5f31879b7dc3","apiVersion":"self-node-remediation.medik8s.io/v1alpha1"},"started":"2026-02-09T15:33:03Z","templateName":"selfnoderemediationtemplate-sample"}]}],"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Storm cooldown completed"},{"type":"StormActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is deactivated, remediation can occur normally"}],"phase":"Remediating","reason":"NHC is remediating 1 nodes","lastUpdateTime":"2026-02-09T15:32:02Z"}, "patch": "{\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2026-02-09T15:05:36Z\",\"message\":\"No issues found, NodeHealthCheck is enabled.\",\"reason\":\"NodeHealthCheckEnabled\",\"status\":\"False\",\"type\":\"Disabled\"},{\"lastTransitionTime\":\"2026-02-09T15:33:03Z\",\"message\":\"Storm cooldown completed\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"False\",\"type\":\"StormCooldownActive\"},{\"lastTransitionTime\":\"2026-02-09T15:33:03Z\",\"message\":\"Storm mode is deactivated, remediation can occur normally\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"False\",\"type\":\"StormActive\"}],\"phase\":\"Remediating\",\"reason\":\"NHC is remediating 1 nodes\",\"unhealthyNodes\":[{\"name\":\"worker-0-2\",\"remediations\":[{\"resource\":{\"apiVersion\":\"self-node-remediation.medik8s.io/v1alpha1\",\"kind\":\"SelfNodeRemediation\",\"name\":\"worker-0-2-jg86c\",\"namespace\":\"openshift-workload-availability\",\"uid\":\"77ad3a6d-39ab-4495-a45a-5f31879b7dc3\"},\"started\":\"2026-02-09T15:33:03Z\",\"templateName\":\"selfnoderemediationtemplate-sample\"}]}]}}"}
2026-02-09T15:33:03.135995203Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:33:03.136052594Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:33:03.13606557Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:33:03.136072372Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:33:03.141314087Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:33:03.141346501Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:33:03.141356961Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:33:03.141362479Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:33:03.172424464Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:33:03.172470299Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:33:03.172483915Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:33:03.172489242Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:33:03.34153156Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "10m0s"}
2026-02-09T15:33:03.341665832Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:33:03.354067021Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:33:03.369928355Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:33:03Z"}
2026-02-09T15:33:03.369973528Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:33:03Z" level=info msg="invalidating lease"
time="2026-02-09T15:33:03Z" level=info msg="getting lease"
2026-02-09T15:33:03.47227686Z	INFO	controllers.NodeHealthCheck	handling unhealthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:33:03.570196662Z	INFO	controllers.NodeHealthCheck.resource manager	external remediation CR already exists	{"CR name": "worker-0-2-jg86c", "kind": "SelfNodeRemediation", "namespace": "openshift-workload-availability"}
time="2026-02-09T15:33:03Z" level=info msg="getting lease"
2026-02-09T15:33:03.570341067Z	INFO	controllers.NodeHealthCheck.nhc lease manager	managing lease - about to try to acquire/extended the lease	{"NodeHealthCheck name": "nhc-worker-test", "lease name": "node-worker-0-2", "NHC is lease owner": true, "lease expiration time": "10m0s"}
2026-02-09T15:33:03.671820414Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "10m0s"}
2026-02-09T15:33:04.182538423Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:33:04.182581892Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:33:04.182594352Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:33:04.182599666Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:33:04.182640729Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:33:04.188644779Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:33:04.194026318Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:33:03Z"}
2026-02-09T15:33:04.194059839Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:33:04Z" level=info msg="invalidating lease"
time="2026-02-09T15:33:04Z" level=info msg="getting lease"
2026-02-09T15:33:04.19728106Z	INFO	controllers.NodeHealthCheck	handling unhealthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:33:04.294595366Z	INFO	controllers.NodeHealthCheck.resource manager	external remediation CR already exists	{"CR name": "worker-0-2-jg86c", "kind": "SelfNodeRemediation", "namespace": "openshift-workload-availability"}
time="2026-02-09T15:33:04Z" level=info msg="getting lease"
2026-02-09T15:33:04.294696864Z	INFO	controllers.NodeHealthCheck.nhc lease manager	managing lease - about to try to acquire/extended the lease	{"NodeHealthCheck name": "nhc-worker-test", "lease name": "node-worker-0-2", "NHC is lease owner": true, "lease expiration time": "10m0s"}
2026-02-09T15:33:04.297582466Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "10m0s"}
2026-02-09T15:34:53.268387765Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-2", "NHC": "nhc-worker-test"}
2026-02-09T15:34:53.268476174Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-2", "NHC": "nhc-worker-test"}
2026-02-09T15:34:53.268531269Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:34:53.276495931Z	INFO	controllers.NodeHealthCheck	Node is going to match unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "False", "duration left": "29.723507642s"}
2026-02-09T15:34:53.28403346Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:33:03Z"}
2026-02-09T15:34:53.284067943Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:34:53Z" level=info msg="invalidating lease"
time="2026-02-09T15:34:53Z" level=info msg="getting lease"
2026-02-09T15:34:53.287283061Z	INFO	controllers.NodeHealthCheck	Ignoring node, because it was unhealthy, and is likely to be unhealthy again.	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:34:53.287681562Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "30.723507642s"}
2026-02-09T15:34:53.692008815Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-2", "NHC": "nhc-worker-test"}
2026-02-09T15:34:53.692094146Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-2", "NHC": "nhc-worker-test"}
2026-02-09T15:34:53.692162754Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:34:53.711490427Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:33:03Z"}
2026-02-09T15:34:53.711541392Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:34:53Z" level=info msg="invalidating lease"
time="2026-02-09T15:34:53Z" level=info msg="getting lease"
2026-02-09T15:34:53.812986324Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:34:53.81996598Z	INFO	controllers.NodeHealthCheck.resource manager	setting a delay for node getting healthy	{"node name": "worker-0-2", "delay in seconds": 5}
2026-02-09T15:34:53.834362967Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:34:53.83442994Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:34:53.834442856Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:34:53.834448727Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:34:53.834689653Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":1,"unhealthyNodes":[{"name":"worker-0-2","remediations":[{"resource":{"kind":"SelfNodeRemediation","namespace":"openshift-workload-availability","name":"worker-0-2-jg86c","uid":"77ad3a6d-39ab-4495-a45a-5f31879b7dc3","apiVersion":"self-node-remediation.medik8s.io/v1alpha1"},"started":"2026-02-09T15:33:03Z","templateName":"selfnoderemediationtemplate-sample"}],"conditionsHealthyTimestamp":"2026-02-09T15:34:53Z","healthyDelayed":true}],"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Storm cooldown completed"},{"type":"StormActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is deactivated, remediation can occur normally"}],"phase":"Remediating","reason":"NHC is remediating 1 nodes","lastUpdateTime":"2026-02-09T15:33:03Z"}, "patch": "{\"status\":{\"unhealthyNodes\":[{\"conditionsHealthyTimestamp\":\"2026-02-09T15:34:53Z\",\"healthyDelayed\":true,\"name\":\"worker-0-2\",\"remediations\":[{\"resource\":{\"apiVersion\":\"self-node-remediation.medik8s.io/v1alpha1\",\"kind\":\"SelfNodeRemediation\",\"name\":\"worker-0-2-jg86c\",\"namespace\":\"openshift-workload-availability\",\"uid\":\"77ad3a6d-39ab-4495-a45a-5f31879b7dc3\"},\"started\":\"2026-02-09T15:33:03Z\",\"templateName\":\"selfnoderemediationtemplate-sample\"}]}]}}"}
2026-02-09T15:34:54.045861585Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "6s"}
2026-02-09T15:34:54.045983605Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:34:54.059894754Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:33:03Z"}
2026-02-09T15:34:54.059933953Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:34:54.161355851Z	INFO	controllers.NodeHealthCheck.resource manager	delaying node getting healthy	{"node name": "worker-0-2", "remaining time in seconds": 3.838649228}
2026-02-09T15:34:54.161420945Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:34:54Z" level=info msg="invalidating lease"
time="2026-02-09T15:34:54Z" level=info msg="getting lease"
2026-02-09T15:34:54.261178238Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "4.838649228s"}
2026-02-09T15:34:59.100388398Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:34:59.112330699Z	INFO	controllers.NodeHealthCheck	found condition	{"NodeHealthCheck name": "nhc-worker-test", "type": "Succeeded", "status": "Unknown", "reason": "", "message": "", "lastTransition": "2026-02-09T15:33:03Z"}
2026-02-09T15:34:59.112379047Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:34:59Z" level=info msg="invalidating lease"
time="2026-02-09T15:34:59Z" level=info msg="getting lease"
2026-02-09T15:34:59.115072853Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:34:59.117661321Z	INFO	controllers.NodeHealthCheck.resource manager	delaying for node getting healthy is done, about to remove the remediation CR	{"node name": "worker-0-2"}
2026-02-09T15:34:59.126460493Z	INFO	controllers.NodeHealthCheck.resource manager	deleted remediation CR	{"name": "worker-0-2-jg86c"}
2026-02-09T15:34:59.126643287Z	DEBUG	events	[remediation] Deleted remediation CR of kind SelfNodeRemediation with name worker-0-2-jg86c	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"145578"}, "reason": "RemediationRemoved"}
2026-02-09T15:34:59.126724576Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:34:59.126763612Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:34:59.126788108Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:34:59.126793868Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:34:59.126808672Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "11s"}
2026-02-09T15:34:59.126900161Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:34:59.213107347Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:34:59Z" level=info msg="invalidating lease"
time="2026-02-09T15:34:59Z" level=info msg="getting lease"
2026-02-09T15:34:59.21553797Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:34:59.217891811Z	INFO	controllers.NodeHealthCheck.resource manager	delaying for node getting healthy is done, about to remove the remediation CR	{"node name": "worker-0-2"}
2026-02-09T15:34:59.220900956Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "17s"}
2026-02-09T15:35:05.016432787Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:35:05.016475797Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:35:05.016493383Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:35:05.016499124Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:35:05.016568179Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:35:05.028035095Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:35:05Z" level=info msg="invalidating lease"
time="2026-02-09T15:35:05Z" level=info msg="getting lease"
2026-02-09T15:35:05.129323311Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:35:05.132340621Z	INFO	controllers.NodeHealthCheck.resource manager	delaying for node getting healthy is done, about to remove the remediation CR	{"node name": "worker-0-2"}
2026-02-09T15:35:05.135453574Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "17s"}
2026-02-09T15:35:10.127950474Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:35:10.140525236Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:35:10Z" level=info msg="invalidating lease"
time="2026-02-09T15:35:10Z" level=info msg="getting lease"
2026-02-09T15:35:10.143519226Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:35:10.146252461Z	INFO	controllers.NodeHealthCheck.resource manager	delaying for node getting healthy is done, about to remove the remediation CR	{"node name": "worker-0-2"}
2026-02-09T15:35:10.1495639Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "17s"}
2026-02-09T15:35:15.299589567Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:35:15.299655076Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:35:15.299680126Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:35:15.299689685Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:35:15.299775106Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:35:15.32447226Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:35:15.42468187Z	INFO	controllers.NodeHealthCheck.resource manager	delaying for node getting healthy is done, about to remove the remediation CR	{"node name": "worker-0-2"}
2026-02-09T15:35:15.428065933Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:35:15Z" level=info msg="invalidating lease"
time="2026-02-09T15:35:15Z" level=info msg="getting lease"
2026-02-09T15:35:15.52605322Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "17s"}
2026-02-09T15:35:16.365350686Z	INFO	controllers.NodeHealthCheck.WatchManager	Request info	{"owner ref": {"apiVersion":"remediation.medik8s.io/v1alpha1","kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","controller":false}}
2026-02-09T15:35:16.365423232Z	INFO	controllers.NodeHealthCheck.WatchManager	mapper: found NHC for remediation CR	{"NHC Name": "nhc-worker-test", "Remediation CR Name": "worker-0-2-jg86c", "Remediation CR Kind": "SelfNodeRemediation"}
2026-02-09T15:35:16.36550502Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:35:16.400856692Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
time="2026-02-09T15:35:16Z" level=info msg="invalidating lease"
time="2026-02-09T15:35:16Z" level=info msg="getting lease"
2026-02-09T15:35:16.506255229Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:35:16Z" level=info msg="invalidating lease"
time="2026-02-09T15:35:16Z" level=info msg="getting lease"
2026-02-09T15:35:16.602520958Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":2,"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Storm cooldown completed"},{"type":"StormActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is deactivated, remediation can occur normally"}],"phase":"Enabled","reason":"NHC is enabled, no ongoing remediation","lastUpdateTime":"2026-02-09T15:34:53Z"}, "patch": "{\"status\":{\"healthyNodes\":2,\"phase\":\"Enabled\",\"reason\":\"NHC is enabled, no ongoing remediation\",\"unhealthyNodes\":null}}"}
2026-02-09T15:35:16.810154461Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "0s"}
2026-02-09T15:35:27.150381898Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:35:27.165240456Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
time="2026-02-09T15:35:27Z" level=info msg="invalidating lease"
time="2026-02-09T15:35:27Z" level=info msg="getting lease"
2026-02-09T15:35:27.16802618Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
time="2026-02-09T15:35:27Z" level=info msg="invalidating lease"
time="2026-02-09T15:35:27Z" level=info msg="getting lease"
2026-02-09T15:35:27.170748468Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "0s"}
2026-02-09T15:39:40.001106074Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-1", "NHC": "nhc-worker-test"}
2026-02-09T15:39:40.001287389Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-1", "NHC": "nhc-worker-test"}
2026-02-09T15:39:40.001462904Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:39:40.014236832Z	INFO	controllers.NodeHealthCheck	Node is going to match unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown", "duration left": "28.985767441s"}
2026-02-09T15:39:40.047684511Z	INFO	controllers.NodeHealthCheck	handling healthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
time="2026-02-09T15:39:40Z" level=info msg="invalidating lease"
time="2026-02-09T15:39:40Z" level=info msg="getting lease"
2026-02-09T15:39:40.148682858Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":1,"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Storm cooldown completed"},{"type":"StormActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is deactivated, remediation can occur normally"}],"phase":"Enabled","reason":"NHC is enabled, no ongoing remediation","lastUpdateTime":"2026-02-09T15:35:16Z"}, "patch": "{\"status\":{\"healthyNodes\":1}}"}
2026-02-09T15:39:40.357993691Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "29.985767441s"}
2026-02-09T15:39:50.744231996Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-2", "NHC": "nhc-worker-test"}
2026-02-09T15:39:50.744414784Z	INFO	adding NHC to reconcile queue for handling node	{"node": "worker-0-2", "NHC": "nhc-worker-test"}
2026-02-09T15:39:50.744515524Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:39:50.753987215Z	INFO	controllers.NodeHealthCheck	Node is going to match unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown", "duration left": "18.246016896s"}
2026-02-09T15:39:50.754113531Z	INFO	controllers.NodeHealthCheck	Node is going to match unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown", "duration left": "29.245887378s"}
2026-02-09T15:39:50.774115233Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":0,"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Storm cooldown completed"},{"type":"StormActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is deactivated, remediation can occur normally"}],"phase":"Enabled","reason":"NHC is enabled, no ongoing remediation","lastUpdateTime":"2026-02-09T15:39:40Z"}, "patch": "{\"status\":{\"healthyNodes\":0}}"}
2026-02-09T15:39:50.98519098Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "19.246016896s"}
2026-02-09T15:40:10.232520422Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:40:10.239252948Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:40:10.23933487Z	INFO	controllers.NodeHealthCheck	Node is going to match unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown", "duration left": "9.760666295s"}
2026-02-09T15:40:10.239434594Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-1", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"148281"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:40:10.246392836Z	INFO	controllers.NodeHealthCheck	Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:40:10.246438674Z	INFO	controllers.NodeHealthCheck	Storm recovery mode activated	{"nhc": "nhc-worker-test"}
2026-02-09T15:40:10.246552229Z	DEBUG	events	[remediation] Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"148281"}, "reason": "RemediationSkipped"}
2026-02-09T15:40:10.246590892Z	DEBUG	events	[remediation] Storm recovery mode activated - delaying remediation until threshold is reached	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"148281"}, "reason": "StormRecoveryStarted"}
2026-02-09T15:40:10.246712478Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":0,"unhealthyNodes":[{"name":"worker-0-1"}],"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Cooldown cleared - storm threshold triggered"},{"type":"StormActive","status":"True","lastTransitionTime":"2026-02-09T15:40:10Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is activated - preventing any new remediation until the storm is over and cooldown duration expired"}],"phase":"Enabled","reason":"NHC is enabled, no ongoing remediation","lastUpdateTime":"2026-02-09T15:39:50Z"}, "patch": "{\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2026-02-09T15:05:36Z\",\"message\":\"No issues found, NodeHealthCheck is enabled.\",\"reason\":\"NodeHealthCheckEnabled\",\"status\":\"False\",\"type\":\"Disabled\"},{\"lastTransitionTime\":\"2026-02-09T15:33:03Z\",\"message\":\"Cooldown cleared - storm threshold triggered\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"False\",\"type\":\"StormCooldownActive\"},{\"lastTransitionTime\":\"2026-02-09T15:40:10Z\",\"message\":\"Storm mode is activated - preventing any new remediation until the storm is over and cooldown duration expired\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"True\",\"type\":\"StormActive\"}],\"unhealthyNodes\":[{\"name\":\"worker-0-1\"}]}}"}
2026-02-09T15:40:10.45762831Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "10.760666295s"}
2026-02-09T15:40:21.219037844Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:40:21.224472835Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:40:21.22453405Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:40:21.224618192Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-1", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"148584"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:40:21.224648521Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"148584"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:40:21.230860843Z	INFO	controllers.NodeHealthCheck	Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:40:21.231016961Z	DEBUG	events	[remediation] Skipped remediation because the number of healthy nodes selected by the selector is 0 and should equal or exceed 1	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"148584"}, "reason": "RemediationSkipped"}
2026-02-09T15:40:21.231281896Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":0,"unhealthyNodes":[{"name":"worker-0-1"},{"name":"worker-0-2"}],"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"False","lastTransitionTime":"2026-02-09T15:33:03Z","reason":"HealthyNodeThresholdChange","message":"Cooldown cleared - storm threshold triggered"},{"type":"StormActive","status":"True","lastTransitionTime":"2026-02-09T15:40:10Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is activated - preventing any new remediation until the storm is over and cooldown duration expired"}],"phase":"Enabled","reason":"NHC is enabled, no ongoing remediation","lastUpdateTime":"2026-02-09T15:40:10Z"}, "patch": "{\"status\":{\"unhealthyNodes\":[{\"name\":\"worker-0-1\"},{\"name\":\"worker-0-2\"}]}}"}
2026-02-09T15:40:21.43915057Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "0s"}
2026-02-09T15:52:48.344649429Z	INFO	nodehealthcheck-resource	validate update	{"name": "nhc-worker-test"}
2026-02-09T15:52:48.34902271Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:52:48.355145345Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:52:48.355243284Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:52:48.355782294Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-1", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"152821"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:52:48.355844239Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"152821"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:52:48.361731771Z	INFO	controllers.NodeHealthCheck	The cluster regained health after the storm, a cooldown period now begins as a safety measure before normal operations resume.	{"nhc": "nhc-worker-test"}
2026-02-09T15:52:48.361799041Z	INFO	controllers.NodeHealthCheck	Storm recovery active: skipping creation of new remediations
2026-02-09T15:52:48.361873675Z	DEBUG	events	[remediation] Storm recovery mode will exit after cooldown delay	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"152821"}, "reason": "StormRecoveryCooldownStarted"}
2026-02-09T15:52:48.361905953Z	DEBUG	events	[remediation] Storm recovery active: skipping creation of new remediations	{"type": "Warning", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"152821"}, "reason": "RemediationSkipped"}
2026-02-09T15:52:48.362107229Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":0,"unhealthyNodes":[{"name":"worker-0-1"},{"name":"worker-0-2"}],"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"True","lastTransitionTime":"2026-02-09T15:52:48Z","reason":"HealthyNodeThresholdChange","message":"Storm cooldown delay started - waiting before resuming normal remediation"},{"type":"StormActive","status":"True","lastTransitionTime":"2026-02-09T15:40:10Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is activated - preventing any new remediation until the storm is over and cooldown duration expired"}],"phase":"Enabled","reason":"NHC is enabled, no ongoing remediation","lastUpdateTime":"2026-02-09T15:40:21Z"}, "patch": "{\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2026-02-09T15:05:36Z\",\"message\":\"No issues found, NodeHealthCheck is enabled.\",\"reason\":\"NodeHealthCheckEnabled\",\"status\":\"False\",\"type\":\"Disabled\"},{\"lastTransitionTime\":\"2026-02-09T15:52:48Z\",\"message\":\"Storm cooldown delay started - waiting before resuming normal remediation\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"True\",\"type\":\"StormCooldownActive\"},{\"lastTransitionTime\":\"2026-02-09T15:40:10Z\",\"message\":\"Storm mode is activated - preventing any new remediation until the storm is over and cooldown duration expired\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"True\",\"type\":\"StormActive\"}]}}"}
2026-02-09T15:52:48.5705822Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "1m1s"}
2026-02-09T15:53:49.571243462Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T15:53:49.578888398Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:53:49.5789434Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T15:53:49.579082557Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-1", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"152824"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:53:49.579114581Z	DEBUG	events	[remediation] Node matches unhealthy condition. Node "worker-0-2", condition type "Ready", condition status "Unknown"	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"152824"}, "reason": "DetectedUnhealthy"}
2026-02-09T15:53:49.584876141Z	INFO	controllers.NodeHealthCheck	Storm recovery mode deactivated	{"nhc": "nhc-worker-test"}
2026-02-09T15:53:49.584935148Z	INFO	controllers.NodeHealthCheck	handling unhealthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T15:53:49.58507905Z	DEBUG	events	[remediation] Storm recovery mode deactivated - normal remediation resumed	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"152824"}, "reason": "StormRecoveryEnded"}
2026-02-09T15:53:49.685541807Z	INFO	controllers.NodeHealthCheck.resource manager	Attempting to obtain Node Lease	{"Node name": "worker-0-1"}
time="2026-02-09T15:53:49Z" level=info msg="request lease"
time="2026-02-09T15:53:49Z" level=info msg="getting lease"
time="2026-02-09T15:53:49Z" level=info msg="create lease"
2026-02-09T15:53:49.689687459Z	INFO	controllers.NodeHealthCheck.resource manager	Creating a remediation CR	{"CR name": "", "CR kind": "SelfNodeRemediation", "namespace": "openshift-workload-availability"}
2026-02-09T15:53:49.700339821Z	DEBUG	events	[remediation] Created remediation object for node worker-0-1	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"152824"}, "reason": "RemediationCreated"}
2026-02-09T15:53:49.702917797Z	INFO	controllers.NodeHealthCheck	handling unhealthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T15:53:49.786013386Z	INFO	controllers.NodeHealthCheck.resource manager	Attempting to obtain Node Lease	{"Node name": "worker-0-2"}
time="2026-02-09T15:53:49Z" level=info msg="request lease"
time="2026-02-09T15:53:49Z" level=info msg="getting lease"
time="2026-02-09T15:53:49Z" level=info msg="create lease"
2026-02-09T15:53:49.789967627Z	INFO	controllers.NodeHealthCheck.resource manager	Creating a remediation CR	{"CR name": "", "CR kind": "SelfNodeRemediation", "namespace": "openshift-workload-availability"}
2026-02-09T15:53:49.796288993Z	DEBUG	events	[remediation] Created remediation object for node worker-0-2	{"type": "Normal", "object": {"kind":"NodeHealthCheck","name":"nhc-worker-test","uid":"c88e2c73-8166-4dd4-bc20-fddbde9e6d3f","apiVersion":"remediation.medik8s.io/v1alpha1","resourceVersion":"152824"}, "reason": "RemediationCreated"}
2026-02-09T15:53:49.800702745Z	INFO	controllers.NodeHealthCheck	Patching NHC status	{"NodeHealthCheck name": "nhc-worker-test", "new status": {"observedNodes":2,"healthyNodes":0,"unhealthyNodes":[{"name":"worker-0-1","remediations":[{"resource":{"kind":"SelfNodeRemediation","namespace":"openshift-workload-availability","name":"worker-0-1-57rvf","uid":"6c198c03-1a13-4adf-8e78-b5f767484d35","apiVersion":"self-node-remediation.medik8s.io/v1alpha1"},"started":"2026-02-09T15:53:49Z","templateName":"selfnoderemediationtemplate-sample"}]},{"name":"worker-0-2","remediations":[{"resource":{"kind":"SelfNodeRemediation","namespace":"openshift-workload-availability","name":"worker-0-2-v5t96","uid":"0d6c2fdc-1d3b-4913-8f09-efbea35aead5","apiVersion":"self-node-remediation.medik8s.io/v1alpha1"},"started":"2026-02-09T15:53:49Z","templateName":"selfnoderemediationtemplate-sample"}]}],"conditions":[{"type":"Disabled","status":"False","lastTransitionTime":"2026-02-09T15:05:36Z","reason":"NodeHealthCheckEnabled","message":"No issues found, NodeHealthCheck is enabled."},{"type":"StormCooldownActive","status":"False","lastTransitionTime":"2026-02-09T15:53:49Z","reason":"HealthyNodeThresholdChange","message":"Storm cooldown completed"},{"type":"StormActive","status":"False","lastTransitionTime":"2026-02-09T15:53:49Z","reason":"HealthyNodeThresholdChange","message":"Storm mode is deactivated, remediation can occur normally"}],"phase":"Remediating","reason":"NHC is remediating 2 nodes","lastUpdateTime":"2026-02-09T15:52:48Z"}, "patch": "{\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2026-02-09T15:05:36Z\",\"message\":\"No issues found, NodeHealthCheck is enabled.\",\"reason\":\"NodeHealthCheckEnabled\",\"status\":\"False\",\"type\":\"Disabled\"},{\"lastTransitionTime\":\"2026-02-09T15:53:49Z\",\"message\":\"Storm cooldown completed\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"False\",\"type\":\"StormCooldownActive\"},{\"lastTransitionTime\":\"2026-02-09T15:53:49Z\",\"message\":\"Storm mode is deactivated, remediation can occur normally\",\"reason\":\"HealthyNodeThresholdChange\",\"status\":\"False\",\"type\":\"StormActive\"}],\"phase\":\"Remediating\",\"reason\":\"NHC is remediating 2 nodes\",\"unhealthyNodes\":[{\"name\":\"worker-0-1\",\"remediations\":[{\"resource\":{\"apiVersion\":\"self-node-remediation.medik8s.io/v1alpha1\",\"kind\":\"SelfNodeRemediation\",\"name\":\"worker-0-1-57rvf\",\"namespace\":\"openshift-workload-availability\",\"uid\":\"6c198c03-1a13-4adf-8e78-b5f767484d35\"},\"started\":\"2026-02-09T15:53:49Z\",\"templateName\":\"selfnoderemediationtemplate-sample\"}]},{\"name\":\"worker-0-2\",\"remediations\":[{\"resource\":{\"apiVersion\":\"self-node-remediation.medik8s.io/v1alpha1\",\"kind\":\"SelfNodeRemediation\",\"name\":\"worker-0-2-v5t96\",\"namespace\":\"openshift-workload-availability\",\"uid\":\"0d6c2fdc-1d3b-4913-8f09-efbea35aead5\"},\"started\":\"2026-02-09T15:53:49Z\",\"templateName\":\"selfnoderemediationtemplate-sample\"}]}]}}"}
2026-02-09T15:53:50.009366063Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "10m0s"}
2026-02-09T16:03:50.010440158Z	INFO	controllers.NodeHealthCheck	reconciling	{"NodeHealthCheck name": "nhc-worker-test"}
2026-02-09T16:03:50.017002278Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-1", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T16:03:50.017041685Z	INFO	controllers.NodeHealthCheck	Node matches unhealthy condition	{"node": "worker-0-2", "condition type": "Ready", "condition status": "Unknown"}
2026-02-09T16:03:50.023106039Z	INFO	controllers.NodeHealthCheck	handling unhealthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-1"}
2026-02-09T16:03:50.028472369Z	INFO	controllers.NodeHealthCheck.resource manager	external remediation CR already exists	{"CR name": "worker-0-1-57rvf", "kind": "SelfNodeRemediation", "namespace": "openshift-workload-availability"}
time="2026-02-09T16:03:50Z" level=info msg="getting lease"
2026-02-09T16:03:50.028564851Z	INFO	controllers.NodeHealthCheck.nhc lease manager	managing lease - about to try to acquire/extended the lease	{"NodeHealthCheck name": "nhc-worker-test", "lease name": "node-worker-0-1", "NHC is lease owner": true, "lease expiration time": "10m0s"}
time="2026-02-09T16:03:50Z" level=info msg="request lease"
time="2026-02-09T16:03:50Z" level=info msg="getting lease"
time="2026-02-09T16:03:50Z" level=info msg="renew lease owned by NodeHealthCheck-nhc-worker-test setAcquireTime=false"
2026-02-09T16:03:50.12421668Z	INFO	controllers.NodeHealthCheck	handling unhealthy node	{"NodeHealthCheck name": "nhc-worker-test", "node": "worker-0-2"}
2026-02-09T16:03:50.130504329Z	INFO	controllers.NodeHealthCheck.resource manager	external remediation CR already exists	{"CR name": "worker-0-2-v5t96", "kind": "SelfNodeRemediation", "namespace": "openshift-workload-availability"}
time="2026-02-09T16:03:50Z" level=info msg="getting lease"
2026-02-09T16:03:50.130601615Z	INFO	controllers.NodeHealthCheck.nhc lease manager	managing lease - about to try to acquire/extended the lease	{"NodeHealthCheck name": "nhc-worker-test", "lease name": "node-worker-0-2", "NHC is lease owner": true, "lease expiration time": "10m0s"}
time="2026-02-09T16:03:50Z" level=info msg="request lease"
time="2026-02-09T16:03:50Z" level=info msg="getting lease"
time="2026-02-09T16:03:50Z" level=info msg="renew lease owned by NodeHealthCheck-nhc-worker-test setAcquireTime=false"
2026-02-09T16:03:50.225646862Z	INFO	controllers.NodeHealthCheck	reconcile end	{"NodeHealthCheck name": "nhc-worker-test", "error": null, "requeue": false, "requeuAfter": "10m0s"}