-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
4.20.0
-
None
-
Quality / Stability / Reliability
-
False
-
-
3
-
Important
-
None
-
None
-
None
-
WINC - Sprint 277
-
1
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
The NodeLogQuery feature gate in kubelet is not honoring its default false value when not explicitly specified in the featureGates configuration given the downstream Openshift fork toggles the default to `true`
https://github.com/openshift/kubernetes/blob/a237dcbeee8bbe1440c94349399cc6eac8870910/pkg/features/kube_features.go#L1543
Resulting on: NodeLogQuery is omitted from the kubeletConfiguration featureGates array, the log query endpoints remain accessible despite the expected default behavior of being disabled.
Version-Release number of selected component (if applicable):
kubelet v1.32.6 (confirmed also in v1.32.5 and v1.31.9)
How reproducible:
Always - occurs consistently across multiple kubelet versions
Steps to Reproduce:
1. Configure kubelet with enableSystemLogQuery: true in kubeletConfiguration
2. Do NOT include NodeLogQuery in the featureGates array (expecting default false behavior)
3. Start kubelet service
4. Test log query endpoints on both Linux and Windows nodes:
- Linux: kubectl get --raw "/api/v1/nodes/linux-node/proxy/logs/?query=sshd&tailLines=1"
- Windows: kubectl get --raw "/api/v1/nodes/win-node/proxy/logs/?query=VSS&tailLines=1"
Actual results:
- Kubelet starts successfully without any validation errors
- Log query endpoints are accessible and return results
- No error message about NodeLogQuery feature gate requirement
- Both Linux and Windows nodes allow log querying despite NodeLogQuery not being explicitly enabled
Below the effective kubelet configuration in a Windows worker node obtained with `oc get --raw "/api/v1/nodes/<node-name>/proxy/configz" | jq . `
The `enableSystemLogQuery` is set, but the `NodeLogQuery` is missing from the `featureGates`, which is inconsistent with upstream documentation and may lead to misunderstanding.
{
"kubeletconfig": {
"enableServer": true,
"podLogsDir": "/var/log/pods",
"syncFrequency": "1m0s",
"fileCheckFrequency": "20s",
"httpCheckFrequency": "20s",
"address": "0.0.0.0",
"port": 10250,
"rotateCertificates": true,
"serverTLSBootstrap": true,
"authentication": {
"x509": {
"clientCAFile": "C:\\k\\kubelet-ca.crt"
},
"webhook": {
"enabled": true,
"cacheTTL": "2m0s"
},
"anonymous": {
"enabled": false
}
},
"authorization": {
"mode": "Webhook",
"webhook": {
"cacheAuthorizedTTL": "5m0s",
"cacheUnauthorizedTTL": "30s"
}
},
"registryPullQPS": 5,
"registryBurst": 10,
"eventRecordQPS": 50,
"eventBurst": 100,
"enableDebuggingHandlers": true,
"healthzPort": 10248,
"healthzBindAddress": "127.0.0.1",
"oomScoreAdj": -999,
"clusterDomain": "cluster.local",
"clusterDNS": [
"172.30.0.10"
],
"streamingConnectionIdleTimeout": "4h0m0s",
"nodeStatusUpdateFrequency": "10s",
"nodeStatusReportFrequency": "5m0s",
"nodeLeaseDurationSeconds": 40,
"imageMinimumGCAge": "2m0s",
"imageMaximumGCAge": "0s",
"imageGCHighThresholdPercent": 85,
"imageGCLowThresholdPercent": 80,
"volumeStatsAggPeriod": "1m0s",
"cgroupsPerQOS": false,
"cgroupDriver": "cgroupfs",
"cpuManagerPolicy": "none",
"cpuManagerReconcilePeriod": "10s",
"memoryManagerPolicy": "None",
"topologyManagerPolicy": "none",
"topologyManagerScope": "container",
"runtimeRequestTimeout": "10m0s",
"hairpinMode": "promiscuous-bridge",
"maxPods": 250,
"podPidsLimit": -1,
"resolvConf": "",
"cpuCFSQuota": true,
"cpuCFSQuotaPeriod": "100ms",
"nodeStatusMaxImages": 50,
"maxOpenFiles": 1000000,
"contentType": "application/vnd.kubernetes.protobuf",
"kubeAPIQPS": 50,
"kubeAPIBurst": 100,
"serializeImagePulls": false,
"evictionHard": {
"imagefs.available": "15%",
"nodefs.available": "10%"
},
"evictionPressureTransitionPeriod": "5m0s",
"enableControllerAttachDetach": true,
"makeIPTablesUtilChains": true,
"iptablesMasqueradeBit": 14,
"iptablesDropBit": 15,
"featureGates": {
"RotateKubeletServerCertificate": true
},
"failSwapOn": true,
"memorySwap": {},
"containerLogMaxSize": "50Mi",
"containerLogMaxFiles": 5,
"containerLogMaxWorkers": 1,
"containerLogMonitorInterval": "10s",
"configMapAndSecretChangeDetectionStrategy": "Watch",
"systemReserved": {
"cpu": "500m",
"ephemeral-storage": "1Gi",
"memory": "2Gi"
},
"enforceNodeAllocatable": [
"none"
],
"volumePluginDir": "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/",
"logging": {
"format": "text",
"flushFrequency": "5s",
"verbosity": 2,
"options": {
"text": {
"infoBufferSize": "0"
},
"json": {
"infoBufferSize": "0"
}
}
},
"enableSystemLogHandler": true,
"enableSystemLogQuery": true,
"shutdownGracePeriod": "0s",
"shutdownGracePeriodCriticalPods": "0s",
"crashLoopBackOff": {},
"enableProfilingHandler": true,
"enableDebugFlagsHandler": true,
"seccompDefault": false,
"memoryThrottlingFactor": 0.9,
"registerWithTaints": [
{
"key": "os",
"value": "Windows",
"effect": "NoSchedule"
}
],
"registerNode": true,
"localStorageCapacityIsolation": true,
"containerRuntimeEndpoint": "npipe://./pipe/containerd-containerd",
"failCgroupV1": false
}
}
Expected results:
- Kubelet should fail to start with validation error when NodeLogQuery is not explicitly set to true
- Should display error: "invalid configuration: NodeLogQuery feature gate is required for enableSystemLogQuery"
- Log query endpoints should be inaccessible
- Behavior should be consistent with explicit NodeLogQuery: false configuration
Additional info:
- According to Kubernetes documentation (https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/), NodeLogQuery defaults to false
- https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/
- causes
-
WINC-1462 Explicitly set NodeLogQuery feature gate when enableSystemLogQuery is true
-
- Closed
-
- links to