Mar 04 13:02:52 podman-etcd(etcd)[246280]: INFO: successfully promoted member '10cff4005fd6585d' (healthy state) Mar 04 13:03:36.815 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[248110] error output [ crm_attribute: Error performing operation: No such device or address ] Mar 04 13:03:36.815 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[248110] error output [ {"level":"warn","ts":"2026-03-04T13:03:24.650137Z","caller":"flags/flag.go:94","msg":"unrecognized environment variable","environment-variable":"ETCDCTL_API=3"} ] Mar 04 13:03:36.815 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[248110] error output [ {"level":"warn","ts":"2026-03-04T13:03:29.654279Z","logger":"etcd-client","caller":"v3@v3.6.5/retry_interceptor.go:65","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00031cd20/10.32.105.80:2379","method":"/etcdserverpb.Cluster/MemberList","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"} ] Mar 04 13:03:36.815 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[248110] error output [ Error: context deadline exceeded ] Mar 04 13:03:36.815 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[248110] error output [ /usr/lib/ocf/resource.d/heartbeat/podman-etcd: line 861: return: : numeric argument required ] Mar 04 13:03:36.815 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[248110] error output [ {"level":"warn","ts":"2026-03-04T13:03:30.974278Z","caller":"flags/flag.go:94","msg":"unrecognized environment variable","environment-variable":"ETCDCTL_API=3"} ] Mar 04 13:03:36.815 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[248110] error output [ {"level":"warn","ts":"2026-03-04T13:03:35.977721Z","logger":"etcd-client","caller":"v3@v3.6.5/retry_interceptor.go:65","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00027cf00/10.32.105.80:2379","method":"/etcdserverpb.Cluster/MemberList","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"} ] Mar 04 13:03:36.815 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[248110] error output [ Error: context deadline exceeded ] Mar 04 13:03:36.815 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[248110] error output [ {"level":"warn","ts":"2026-03-04T13:03:36.509419Z","caller":"flags/flag.go:94","msg":"unrecognized environment variable","environment-variable":"ETCDCTL_API=3"} ] Mar 04 13:04:12.090 ocp2-cp-0 pacemaker-controld [42728] (process_graph_event) notice: Transition 291 action 21 (etcd_monitor_30000 on ocp2-cp-1): expected 'ok' but got 'error' | target-rc=0 rc=1 call-id=24 event='arrived after initial scheduling' Mar 04 13:04:12.156 ocp2-cp-0 pacemaker-schedulerd[42727] (unpack_rsc_op_failure) warning: Unexpected result (error: Resource agent did not complete within 25s) was recorded for monitor of etcd:1 on ocp2-cp-1 at Mar 4 13:04:12 2026 | exit-status=1 id=etcd_last_failure_0 Mar 04 13:04:20.180 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[249944] error output [ crm_attribute: Error performing operation: No such device or address ] Mar 04 13:04:20.180 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[249944] error output [ {"level":"warn","ts":"2026-03-04T13:04:08.047966Z","caller":"flags/flag.go:94","msg":"unrecognized environment variable","environment-variable":"ETCDCTL_API=3"} ] Mar 04 13:04:20.180 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[249944] error output [ {"level":"warn","ts":"2026-03-04T13:04:13.050392Z","logger":"etcd-client","caller":"v3@v3.6.5/retry_interceptor.go:65","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0002fcf00/10.32.105.80:2379","method":"/etcdserverpb.Cluster/MemberList","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"} ] Mar 04 13:04:20.180 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[249944] error output [ Error: context deadline exceeded ] Mar 04 13:04:20.180 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[249944] error output [ /usr/lib/ocf/resource.d/heartbeat/podman-etcd: line 861: return: : numeric argument required ] Mar 04 13:04:20.180 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[249944] error output [ {"level":"warn","ts":"2026-03-04T13:04:14.066576Z","caller":"flags/flag.go:94","msg":"unrecognized environment variable","environment-variable":"ETCDCTL_API=3"} ] Mar 04 13:04:20.180 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[249944] error output [ {"level":"warn","ts":"2026-03-04T13:04:19.069052Z","logger":"etcd-client","caller":"v3@v3.6.5/retry_interceptor.go:65","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0002fcf00/10.32.105.80:2379","method":"/etcdserverpb.Cluster/MemberList","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"} ] Mar 04 13:04:20.180 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[249944] error output [ Error: context deadline exceeded ] Mar 04 13:04:20.180 ocp2-cp-0 pacemaker-execd [42725] (log_op_output) info: etcd_monitor_30000[249944] error output [ {"level":"warn","ts":"2026-03-04T13:04:19.889548Z","caller":"flags/flag.go:94","msg":"unrecognized environment variable","environment-variable":"ETCDCTL_API=3"} ] Mar 04 13:04:20 podman-etcd(etcd)[250725]: INFO: notify: type=pre, operation=stop, nodes { active=[ocp2-cp-0 ocp2-cp-1], start=[ocp2-cp-1], stop=[ocp2-cp-1] }, resources { active=[etcd:0 etcd:1], start =[etcd:1], stop=[etcd:1] } Mar 04 13:04:20.426 ocp2-cp-0 pacemaker-controld [42728] (execute_rsc_action) notice: Initiating stop operation etcd_stop_0 on ocp2-cp-1 | action 7 Mar 04 13:04:54 podman-etcd(etcd)[251815]: INFO: ocp2-cp-1 is not in the members list Mar 04 13:04:54 podman-etcd(etcd)[251815]: INFO: add ocp2-cp-1 (10.32.105.81) to the member list as learner Mar 04 13:04:54 podman-etcd(etcd)[251815]: INFO: Member 6824b1507d5ee97d added as learner to cluster a8a5995d22e1235d Mar 04 13:05:50.435 ocp2-cp-0 pacemaker-based [42723] (cib_perform_op) info: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='etcd']/lrm_rsc_op[@id='etcd_last_0']: @transition-magic=2:1;7:299:0:6682ed82-38b2-49e6-b434-60ea3c5decb8, @exit-reason=Resource agent did not complete within 1m30s, @call-id=27, @rc-code=1, @op-status=2, @exec-time=90003 Mar 04 13:05:50.438 ocp2-cp-0 pacemaker-controld [42728] (update_failcount) info: Updating failcount for etcd on ocp2-cp-1 after failed stop: rc=1 (update=INFINITY, time=1772629550) Mar 04 13:05:50.438 ocp2-cp-0 pacemaker-controld [42728] (process_graph_event) notice: Transition 299 action 7 (etcd_stop_0 on ocp2-cp-1): expected 'ok' but got 'error' | target-rc=0 rc=1 call-id=27 Mar 04 13:05:50.439 ocp2-cp-0 pacemaker-controld [42728] (execute_rsc_action) notice: Initiating notify operation etcd_post_notify_stop_0 locally on ocp2-cp-0 | action 39 Mar 04 13:05:50.439 ocp2-cp-0 pacemaker-controld [42728] (do_lrm_rsc_op) notice: Requesting local execution of notify operation for etcd on ocp2-cp-0 | transition_key=39:299:0:6682ed82-38b2-49e6-b434-60ea3c5decb8 op_key=etcd_notify_0 Mar 04 13:05:50.440 ocp2-cp-0 pacemaker-execd [42725] (log_execute) info: executing - rsc:etcd action:notify call_id:63 Mar 04 13:05:50.443 ocp2-cp-0 pacemaker-attrd [42726] (update_attr_on_host) notice: Setting last-failure-etcd#stop_0[ocp2-cp-1] in instance_attributes: (unset) -> 1772629550 | from ocp2-cp-0 with no write delay and node XML ID 2 Mar 04 13:05:50.443 ocp2-cp-0 pacemaker-attrd [42726] (write_attribute) info: Sent CIB request 53 with 1 change for last-failure-etcd#stop_0 (set unspecified) Mar 04 13:05:50.443 ocp2-cp-0 pacemaker-attrd [42726] (update_attr_on_host) notice: Setting fail-count-etcd#stop_0[ocp2-cp-1] in instance_attributes: (unset) -> INFINITY | from ocp2-cp-0 with no write delay and node XML ID 2 Mar 04 13:05:50.626 ocp2-cp-0 pacemaker-schedulerd[42727] (unpack_rsc_op_failure) warning: Unexpected result (error: Resource agent did not complete within 1m30s) was recorded for stop of etcd:1 on ocp2-cp-1 at Mar 4 13:04:20 2026 | exit-status=1 id=etcd_last_failure_0 Mar 04 13:05:50.626 ocp2-cp-0 pacemaker-schedulerd[42727] (unpack_rsc_op_failure) warning: Unexpected result (error: Resource agent did not complete within 1m30s) was recorded for stop of etcd:1 on ocp2-cp-1 at Mar 4 13:04:20 2026 | exit-status=1 id=etcd_last_0 Mar 04 13:05:50.626 ocp2-cp-0 pacemaker-schedulerd[42727] (pe_fence_node) warning: Cluster node ocp2-cp-1 will be fenced: etcd:1 failed there Mar 04 13:05:50.629 ocp2-cp-0 pacemaker-schedulerd[42727] (log_list_item) notice: Actions: Fence (reboot) ocp2-cp-1 'etcd:1 failed there' Mar 04 13:10:23.400 ocp2-cp-0 pacemaker-controld [42728] (update_failcount) info: Updating failcount for etcd on ocp2-cp-0 after failed monitor: rc=1 (update=value++, time=1772629823) Mar 04 13:10:23.401 ocp2-cp-0 pacemaker-attrd [42726] (handle_value_expansion) info: Expanded fail-count-etcd#monitor_30000=value++ to 2 Mar 04 13:10:23.404 ocp2-cp-0 pacemaker-controld [42728] (process_graph_event) notice: Transition 286 action 16 (etcd_monitor_30000 on ocp2-cp-0): expected 'ok' but got 'error' | target-rc=0 rc=1 call-id=57 event='arrived after initial scheduling' Mar 04 13:10:23.404 ocp2-cp-0 pacemaker-attrd [42726] (update_attr_on_host) notice: Setting last-failure-etcd#monitor_30000[ocp2-cp-0] in instance_attributes: 1772629090 -> 1772629823 | from ocp2-cp-0 with no write delay and node XML ID 1 Mar 04 13:10:23.758 ocp2-cp-0 pacemaker-schedulerd[42727] (unpack_rsc_op_failure) warning: Unexpected result (error: Resource agent did not complete within 25s) was recorded for monitor of etcd:0 on ocp2-cp-0 at Mar 4 13:10:23 2026 | exit-status=1 id=etcd_last_failure_0 Mar 04 13:10:23.763 ocp2-cp-0 pacemaker-schedulerd[42727] (log_list_item) notice: Actions: Recover etcd:0 ( ocp2-cp-0 ) Mar 04 13:11:54.792 ocp2-cp-0 pacemaker-execd [42725] (async_action_complete) info: Resource agent etcd_stop_0[273405] timed out after 1m30s Mar 04 13:11:54.794 ocp2-cp-0 pacemaker-controld [42728] (log_executor_event) notice: etcd_stop_0@ocp2-cp-0 output [ Deleted crm_config option: id=cib-bootstrap-options-learner_node name=learner_node ] Mar 04 13:11:54.799 ocp2-cp-0 pacemaker-based [42723] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='etcd']/lrm_rsc_op[@id='etcd_last_failure_0']: @operation_key=etcd_stop_0, @operation=stop, @transition-key=2:309:0:6682ed82-38b2-49e6-b434-60ea3c5decb8, @transition-magic=2:1;2:309:0:6682ed82-38b2-49e6-b434-60ea3c5decb8, @exit-reason=Resource agent did not complete within 1m30s, @call-id=68, @interval=0, @last-rc-change=1772629824, @exec-time=90005, @op-digest=fa918ccc21f6b7ebd7b01b8e7bba84eb Mar 04 13:11:54.799 ocp2-cp-0 pacemaker-based [42723] (cib_perform_op) info: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='etcd']/lrm_rsc_op[@id='etcd_last_0']: @transition-magic=2:1;2:309:0:6682ed82-38b2-49e6-b434-60ea3c5decb8, @exit-reason=Resource agent did not complete within 1m30s, @call-id=68, @rc-code=1, @op-status=2, @exec-time=90005 Mar 04 13:11:54.800 ocp2-cp-0 pacemaker-controld [42728] (abort_transition_graph) info: Transition 309 aborted by operation etcd_stop_0 'modify' on ocp2-cp-0: Event failed | magic=2:1;2:309:0:6682ed82-38b2-49e6-b434-60ea3c5decb8 cib=0.331.1 source=process_graph_event:558 complete=false Mar 04 13:11:54.800 ocp2-cp-0 pacemaker-controld [42728] (update_failcount) info: Updating failcount for etcd on ocp2-cp-0 after failed stop: rc=1 (update=INFINITY, time=1772629914) Mar 04 13:11:54.801 ocp2-cp-0 pacemaker-controld [42728] (process_graph_event) notice: Transition 309 action 2 (etcd_stop_0 on ocp2-cp-0): expected 'ok' but got 'error' | target-rc=0 rc=1 call-id=68 Mar 04 13:11:54.801 ocp2-cp-0 pacemaker-attrd [42726] (update_attr_on_host) notice: Setting fail-count-etcd#stop_0[ocp2-cp-0] in instance_attributes: (unset) -> INFINITY | from ocp2-cp-0 with no write delay and node XML ID 1 Mar 04 13:11:54.817 ocp2-cp-0 pacemaker-attrd [42726] (attrd_cib_callback) info: * Wrote fail-count-etcd#stop_0[ocp2-cp-0]=INFINITY Mar 04 13:11:54.850 ocp2-cp-0 pacemaker-schedulerd[42727] (unpack_rsc_op_failure) warning: Unexpected result (error: Resource agent did not complete within 1m30s) was recorded for stop of etcd:0 on ocp2-cp-0 at Mar 4 13:10:24 2026 | exit-status=1 id=etcd_last_0 Mar 04 13:11:54.850 ocp2-cp-0 pacemaker-schedulerd[42727] (unpack_rsc_op_failure) warning: Unexpected result (error: Resource agent did not complete within 1m30s) was recorded for stop of etcd:0 on ocp2-cp-0 at Mar 4 13:10:24 2026 | exit-status=1 id=etcd_last_failure_0 Mar 04 13:11:54.850 ocp2-cp-0 pacemaker-schedulerd[42727] (pe_fence_node) warning: Cluster node ocp2-cp-0 will be fenced: etcd:0 failed there Mar 04 13:11:54.852 ocp2-cp-0 pacemaker-schedulerd[42727] (schedule_fencing) warning: Scheduling node ocp2-cp-0 for fencing Mar 04 13:11:54.853 ocp2-cp-0 pacemaker-schedulerd[42727] (log_list_item) notice: Actions: Fence (reboot) ocp2-cp-0 'etcd:0 failed there'