Details
-
Bug
-
Resolution: Done
-
Undefined
-
4.13, 4.12
-
False
-
-
N/A
-
Bug Fix
-
Done
Description
This is a clone of issue OCPBUGS-7149. The following is the description of the original issue:
—
Description of problem:
When deploying an spoke multinode ipv6 cluster. The cluster dont continue installation while waiting for rebooting nodes. But, these were rebooted.
both mater-0 and mater-1 are rebooted.
In the bootstrap I can see many errors:
time="2023-02-07T10:53:23Z" level=info msg="Found 0 ready master nodes" time="2023-02-07T10:53:27Z" level=info msg="searching for hosts that pulled ignition already" time="2023-02-07T10:53:27Z" level=info msg="Verifying if host master-0.el8k-ztp-1.hpecloud.org pulled ignition" time="2023-02-07T10:53:27Z" level=info msg="Verifying if host master-1.el8k-ztp-1.hpecloud.org pulled ignition" master-1.el8k-ztp-1.hpecloud.org Writing image to disk master-0.el8k-ztp-1.hpecloud.org Writing image to disk time="2023-02-07T10:53:28Z" level=info msg="Found 1 master nodes: map[master-0.el8k-ztp-1.hpecloud.org:[{Type:MemoryPressure Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.} {Type:DiskPressure Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.} {Type:PIDPressure Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.} {Type:Ready Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.}]]" time="2023-02-07T10:53:28Z" level=info msg="Found 0 ready master nodes" time="2023-02-07T10:53:33Z" level=info msg="Found 1 master nodes: map[master-0.el8k-ztp-1.hpecloud.org:[{Type:MemoryPressure Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.} {Type:DiskPressure Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.} {Type:PIDPressure Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.} {Type:Ready Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.}]]" time="2023-02-07T10:53:33Z" level=info msg="Found 0 ready master nodes" time="2023-02-07T10:53:38Z" level=info msg="Found 1 master nodes: map[master-0.el8k-ztp-1.hpecloud.org:[{Type:MemoryPressure Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.} {Type:DiskPressure Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.} {Type:PIDPressure Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.} {Type:Ready Status:Unknown LastHeartbeatTime:2023-02-07 10:41:23 +0000 UTC LastTransitionTime:2023-02-07 10:42:41 +0000 UTC Reason:NodeStatusUnknown Message:Kubelet stopped posting node status.}]]" time="2023-02-07T10:53:38Z" level=info msg="Found 0 ready master nodes"
it seems a problem related to IPv6
This is the MCS logs from the bootstrap node:
I0206 16:23:40.963203 1 bootstrap.go:37] Version: v4.12.0-202301171436.p0.g60fb64f.assembly.stream-dirty (60fb64fa861f1231f779226aa5dcd5c4ef604dcb) I0206 16:23:40.963305 1 api.go:64] Launching server on :22624 I0206 16:23:40.963321 1 api.go:64] Launching server on :22623 I0206 16:26:48.364205 1 api.go:109] Pool master requested by address:"[2620:52:0:1351:67c2:adb3:cfd6:83]:47018" User-Agent:"Ignition/2.14.0" Accept-Header: "application/vnd.coreos.ignition+json;version=3.3.0, */*;q=0.1" I0206 16:26:48.364233 1 bootstrap_server.go:66] reading file "/etc/mcs/bootstrap/machine-pools/master.yaml" I0206 16:26:48.365296 1 bootstrap_server.go:86] reading file "/etc/mcs/bootstrap/machine-configs/rendered-master-2a58b078298d4e5124ad04da186bff14.yaml" I0206 16:45:53.384522 1 api.go:109] Pool master requested by address:"[2620:52:0:1351:67c2:adb3:cfd6:85]:49690" User-Agent:"Ignition/2.14.0" Accept-Header: "application/vnd.coreos.ignition+json;version=3.3.0, */*;q=0.1" I0206 16:45:53.384560 1 bootstrap_server.go:66] reading file "/etc/mcs/bootstrap/machine-pools/master.yaml" I0206 16:45:53.385438 1 bootstrap_server.go:86] reading file "/etc/mcs/bootstrap/machine-configs/rendered-master-2a58b078298d4e5124ad04da186bff14.yaml"
Version-Release number of selected component (if applicable):
4.12
How reproducible:
Steps to Reproduce:
1.Create new spoke cluster only ipv6 2.Nodes start the installation 3.Wait for nodes to pull ignition 4.The nodes reboot and pull ignition
Actual results:
The UI still says that they're rebooting
Expected results:
The UI should say the nodes are in the configuring stage
Additional info:
The issue is due to a bad MCS log parsing regex not detecing IPv6 bracketed addresses