May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: handle_request: Client stonith_admin.11395.c4ac81ce wants to fence (reboot) 'vmhost1-fsl.bcn' with device '(any)' May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: initiate_remote_stonith_op: Requesting peer fencing (reboot) of vmhost1-fsl.bcn | id=11676486-cc58-4ac5-baae-f015cd660998 state=0 May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: can_fence_host_with_device: fence_vmhost0_ipmi can not fence (reboot) vmhost1-fsl.bcn: static-list May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: can_fence_host_with_device: fence_vmhost1_ipmi can fence (reboot) vmhost1-fsl.bcn: static-list May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: process_remote_stonith_query: Query result 1 of 2 from vmhost0-fsl.bcn for vmhost1-fsl.bcn/reboot (1 devices) 11676486-cc58-4ac5-baae-f015cd660998 May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: process_remote_stonith_query: Query result 2 of 2 from vmhost1-fsl.bcn for vmhost1-fsl.bcn/reboot (1 devices) 11676486-cc58-4ac5-baae-f015cd660998 May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: process_remote_stonith_query: All query replies have arrived, continuing (2 expected/2 received) May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: call_remote_stonith: Total timeout set to 120 for peer's fencing of vmhost1-fsl.bcn for stonith_admin.11395|id=11676486-cc58-4ac5-baae-f015cd660998 May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: call_remote_stonith: Requesting that 'vmhost0-fsl.bcn' perform op 'vmhost1-fsl.bcn reboot' for stonith_admin.11395 (144s, 0s) May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: can_fence_host_with_device: fence_vmhost0_ipmi can not fence (reboot) vmhost1-fsl.bcn: static-list May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: can_fence_host_with_device: fence_vmhost1_ipmi can fence (reboot) vmhost1-fsl.bcn: static-list May 10 15:25:09 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: stonith_fence_get_devices_cb: Found 1 matching devices for 'vmhost1-fsl.bcn' May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: crm_update_peer_expected: handle_request: Node vmhost1-fsl.bcn[2] - expected state is now down (was member) May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: handle_shutdown_request: Creating shutdown request for vmhost1-fsl.bcn (state=S_IDLE) May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: update_attrd_helper: Connecting to attribute manager ... 5 retries remaining May 10 15:25:13 [11142] vmhost0-fsl.jsc.nasa.gov attrd: info: attrd_peer_update: Setting shutdown[vmhost1-fsl.bcn]: (null) -> 1525983913 from vmhost0-fsl.bcn May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: --- 0.13.20 2 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: +++ 0.13.21 (null) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib: @num_updates=21 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: ++ /cib/status/node_state[@id='2']: May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: ++ May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: ++ May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: ++ May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: ++ May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vmhost1-fsl.bcn/attrd/4, version=0.13.21) May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: abort_transition_graph: Transition aborted by transient_attributes.2 'create': Transient attribute change | cib=0.13.21 source=abort_unless_down:343 path=/cib/status/node_state[@id='2'] complete=true May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: notice: unpack_config: On loss of CCM Quorum: Ignore May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: determine_online_status_fencing: Node vmhost0-fsl.bcn is active May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: determine_online_status: Node vmhost0-fsl.bcn is online May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: determine_online_status: Node vmhost1-fsl.bcn is shutting down May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: unpack_node_loop: Node 1 is already processed May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: unpack_node_loop: Node 2 is already processed May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: unpack_node_loop: Node 1 is already processed May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: unpack_node_loop: Node 2 is already processed May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: common_print: fence_vmhost0_ipmi (stonith:fence_ipmilan): Started vmhost0-fsl.bcn May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: common_print: fence_vmhost1_ipmi (stonith:fence_ipmilan): Started vmhost1-fsl.bcn May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: RecurringOp: Start recurring monitor (60s) for fence_vmhost1_ipmi on vmhost0-fsl.bcn May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: notice: stage6: Scheduling Node vmhost1-fsl.bcn for shutdown May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: notice: LogNodeActions: * Shutdown vmhost1-fsl.bcn May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: LogActions: Leave fence_vmhost0_ipmi (Started vmhost0-fsl.bcn) May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: notice: LogAction: * Move fence_vmhost1_ipmi ( vmhost1-fsl.bcn -> vmhost0-fsl.bcn ) May 10 15:25:13 [11143] vmhost0-fsl.jsc.nasa.gov pengine: notice: process_pe_message: Calculated transition 1, saving inputs in /var/lib/pacemaker/pengine/pe-input-24.bz2 May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: do_te_invoke: Processing graph 1 (ref=pe_calc-dc-1525983913-22) derived from /var/lib/pacemaker/pengine/pe-input-24.bz2 May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: notice: te_rsc_command: Initiating stop operation fence_vmhost1_ipmi_stop_0 on vmhost1-fsl.bcn | action 6 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: --- 0.13.21 2 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: +++ 0.13.22 (null) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib: @num_updates=22 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='fence_vmhost1_ipmi']/lrm_rsc_op[@id='fence_vmhost1_ipmi_last_0']: @operation_key=fence_vmhost1_ipmi_stop_0, @operation=stop, @transition-key=6:1:0:15826900-16df-4cd4-9f7d-7213b59d15a6, @transition-magic=-1:193;6:1:0:15826900-16df-4cd4-9f7d-7213b59d15a6, @call-id=-1, @rc-code=193, @op-status=-1, @last-run=1525983913, @last-rc-change=152598391 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vmhost1-fsl.bcn/crmd/18, version=0.13.22) May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: update_cib_stonith_devices_v2: Updating device list from the cib: modify lrm_rsc_op[@id='fence_vmhost1_ipmi_last_0'] May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: cib_devices_update: Updating devices to version 0.13.22 May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: unpack_config: On loss of CCM Quorum: Ignore May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: --- 0.13.22 2 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: +++ 0.13.23 (null) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib: @num_updates=23 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='fence_vmhost1_ipmi']/lrm_rsc_op[@id='fence_vmhost1_ipmi_last_0']: @transition-magic=0:0;6:1:0:15826900-16df-4cd4-9f7d-7213b59d15a6, @call-id=13, @rc-code=0, @op-status=0 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vmhost1-fsl.bcn/crmd/19, version=0.13.23) May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: match_graph_event: Action fence_vmhost1_ipmi_stop_0 (6) confirmed on vmhost1-fsl.bcn (rc=0) May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: notice: te_rsc_command: Initiating start operation fence_vmhost1_ipmi_start_0 locally on vmhost0-fsl.bcn | action 7 May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: do_lrm_rsc_op: Performing key=7:1:0:15826900-16df-4cd4-9f7d-7213b59d15a6 op=fence_vmhost1_ipmi_start_0 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Forwarding cib_modify operation for section status to all (origin=local/crmd/42) May 10 15:25:13 [11141] vmhost0-fsl.jsc.nasa.gov lrmd: info: log_execute: executing - rsc:fence_vmhost1_ipmi action:start call_id:12 May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: te_crm_command: Executing crm-event (10): do_shutdown on vmhost1-fsl.bcn May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: update_cib_stonith_devices_v2: Updating device list from the cib: modify lrm_rsc_op[@id='fence_vmhost1_ipmi_last_0'] May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: cib_devices_update: Updating devices to version 0.13.23 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: --- 0.13.23 2 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: +++ 0.13.24 (null) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib: @num_updates=24 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='fence_vmhost1_ipmi']/lrm_rsc_op[@id='fence_vmhost1_ipmi_last_0']: @operation_key=fence_vmhost1_ipmi_start_0, @operation=start, @transition-key=7:1:0:15826900-16df-4cd4-9f7d-7213b59d15a6, @transition-magic=-1:193;7:1:0:15826900-16df-4cd4-9f7d-7213b59d15a6, @call-id=-1, @rc-code=193, @op-status=-1, @last-run=1525983913, @last-rc-change=1525983 May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: unpack_config: On loss of CCM Quorum: Ignore May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vmhost0-fsl.bcn/crmd/42, version=0.13.24) May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: pcmk_cpg_membership: Node 2 left group crmd (peer=vmhost1-fsl.bcn, counter=2.0) May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: crm_update_peer_proc: pcmk_cpg_membership: Node vmhost1-fsl.bcn[2] - corosync-cpg is now offline May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: peer_update_callback: Client vmhost1-fsl.bcn/peer now has status [offline] (DC=true, changed=4000000) May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: peer_update_callback: Peer vmhost1-fsl.bcn left us May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: erase_status_tag: Deleting transient_attributes status entries for vmhost1-fsl.bcn | xpath=//node_state[@uname='vmhost1-fsl.bcn']/transient_attributes May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Forwarding cib_delete operation for section //node_state[@uname='vmhost1-fsl.bcn']/transient_attributes to all (origin=local/crmd/43) May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: notice: peer_update_callback: do_shutdown of peer vmhost1-fsl.bcn is complete | op=10 May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: crm_update_peer_join: crmd_peer_down: Node vmhost1-fsl.bcn[2] - join-2 phase confirmed -> none May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: pcmk_cpg_membership: Node 1 still member of group crmd (peer=vmhost0-fsl.bcn, counter=2.0) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Forwarding cib_modify operation for section status to all (origin=local/crmd/44) May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: update_cib_stonith_devices_v2: Updating device list from the cib: modify lrm_rsc_op[@id='fence_vmhost1_ipmi_last_0'] May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: cib_devices_update: Updating devices to version 0.13.24 May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: unpack_config: On loss of CCM Quorum: Ignore May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: --- 0.13.24 2 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: +++ 0.13.25 bb7eb20c2878d573f8b81b8aed19e68f May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: -- /cib/status/node_state[@id='2']/transient_attributes[@id='2'] May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib: @num_updates=25 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='vmhost1-fsl.bcn']/transient_attributes: OK (rc=0, origin=vmhost0-fsl.bcn/crmd/43, version=0.13.25) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: --- 0.13.25 2 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: +++ 0.13.26 (null) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib: @num_updates=26 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib/status/node_state[@id='2']: @crmd=offline, @crm-debug-origin=peer_update_callback, @join=down, @expected=down May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vmhost0-fsl.bcn/crmd/44, version=0.13.26) May 10 15:25:13 [11142] vmhost0-fsl.jsc.nasa.gov attrd: info: pcmk_cpg_membership: Node 2 left group attrd (peer=vmhost1-fsl.bcn, counter=2.0) May 10 15:25:13 [11142] vmhost0-fsl.jsc.nasa.gov attrd: info: crm_update_peer_proc: pcmk_cpg_membership: Node vmhost1-fsl.bcn[2] - corosync-cpg is now offline May 10 15:25:13 [11142] vmhost0-fsl.jsc.nasa.gov attrd: notice: crm_update_peer_state_iter: Node vmhost1-fsl.bcn state is now lost | nodeid=2 previous=member source=crm_update_peer_proc May 10 15:25:13 [11142] vmhost0-fsl.jsc.nasa.gov attrd: notice: attrd_peer_remove: Removing all vmhost1-fsl.bcn attributes for peer loss May 10 15:25:13 [11142] vmhost0-fsl.jsc.nasa.gov attrd: notice: attrd_peer_change_cb: Lost attribute writer vmhost1-fsl.bcn May 10 15:25:13 [11142] vmhost0-fsl.jsc.nasa.gov attrd: info: crm_reap_dead_member: Removing node with name vmhost1-fsl.bcn and id 2 from membership cache May 10 15:25:13 [11142] vmhost0-fsl.jsc.nasa.gov attrd: notice: reap_crm_member: Purged 1 peer with id=2 and/or uname=vmhost1-fsl.bcn from the membership cache May 10 15:25:13 [11142] vmhost0-fsl.jsc.nasa.gov attrd: info: pcmk_cpg_membership: Node 1 still member of group attrd (peer=vmhost0-fsl.bcn, counter=2.0) May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: pcmk_cpg_membership: Node 2 left group stonith-ng (peer=vmhost1-fsl.bcn, counter=2.0) May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: crm_update_peer_proc: pcmk_cpg_membership: Node vmhost1-fsl.bcn[2] - corosync-cpg is now offline May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_shutdown_req: Shutdown REQ from vmhost1-fsl.bcn May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: crm_update_peer_state_iter: Node vmhost1-fsl.bcn state is now lost | nodeid=2 previous=member source=crm_update_peer_proc May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: crm_reap_dead_member: Removing node with name vmhost1-fsl.bcn and id 2 from membership cache May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: reap_crm_member: Purged 1 peer with id=2 and/or uname=vmhost1-fsl.bcn from the membership cache May 10 15:25:13 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: info: pcmk_cpg_membership: Node 1 still member of group stonith-ng (peer=vmhost0-fsl.bcn, counter=2.0) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: pcmk_cpg_membership: Node 2 left group cib (peer=vmhost1-fsl.bcn, counter=2.0) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: crm_update_peer_proc: pcmk_cpg_membership: Node vmhost1-fsl.bcn[2] - corosync-cpg is now offline May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: notice: crm_update_peer_state_iter: Node vmhost1-fsl.bcn state is now lost | nodeid=2 previous=member source=crm_update_peer_proc May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: crm_reap_dead_member: Removing node with name vmhost1-fsl.bcn and id 2 from membership cache May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: notice: reap_crm_member: Purged 1 peer with id=2 and/or uname=vmhost1-fsl.bcn from the membership cache May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: pcmk_cpg_membership: Node 1 still member of group cib (peer=vmhost0-fsl.bcn, counter=2.0) May 10 15:25:13 [11138] vmhost0-fsl.jsc.nasa.gov pacemakerd: info: pcmk_cpg_membership: Node 2 left group pacemakerd (peer=vmhost1-fsl.bcn, counter=2.0) May 10 15:25:13 [11138] vmhost0-fsl.jsc.nasa.gov pacemakerd: info: crm_update_peer_proc: pcmk_cpg_membership: Node vmhost1-fsl.bcn[2] - corosync-cpg is now offline May 10 15:25:13 [11138] vmhost0-fsl.jsc.nasa.gov pacemakerd: info: pcmk_cpg_membership: Node 1 still member of group pacemakerd (peer=vmhost0-fsl.bcn, counter=2.0) May 10 15:25:13 [11138] vmhost0-fsl.jsc.nasa.gov pacemakerd: info: mcp_cpg_deliver: Ignoring process list sent by peer for local node [11130] vmhost0-fsl.jsc.nasa.gov corosyncnotice [TOTEM ] A new membership (192.168.1.140:184) was formed. Members left: 2 [11130] vmhost0-fsl.jsc.nasa.gov corosyncnotice [QUORUM] Members[1]: 1 May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: pcmk_quorum_notification: Quorum retained | membership=184 members=1 [11130] vmhost0-fsl.jsc.nasa.gov corosyncnotice [MAIN ] Completed service synchronization, ready to provide service. May 10 15:25:13 [11138] vmhost0-fsl.jsc.nasa.gov pacemakerd: info: pcmk_quorum_notification: Quorum retained | membership=184 members=1 May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: notice: crm_update_peer_state_iter: Node vmhost1-fsl.bcn state is now lost | nodeid=2 previous=member source=crm_reap_unseen_nodes May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: peer_update_callback: Cluster node vmhost1-fsl.bcn is now lost (was member) May 10 15:25:13 [11138] vmhost0-fsl.jsc.nasa.gov pacemakerd: notice: crm_update_peer_state_iter: Node vmhost1-fsl.bcn state is now lost | nodeid=2 previous=member source=crm_reap_unseen_nodes May 10 15:25:13 [11144] vmhost0-fsl.jsc.nasa.gov crmd: notice: peer_update_callback: do_shutdown of peer vmhost1-fsl.bcn is complete | op=10 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Forwarding cib_modify operation for section status to all (origin=local/crmd/45) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Forwarding cib_modify operation for section nodes to all (origin=local/crmd/48) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Forwarding cib_modify operation for section status to all (origin=local/crmd/49) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vmhost0-fsl.bcn/crmd/45, version=0.13.26) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=vmhost0-fsl.bcn/crmd/48, version=0.13.26) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: --- 0.13.26 2 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: Diff: +++ 0.13.27 (null) May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib: @num_updates=27 May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib/status/node_state[@id='1']: @crm-debug-origin=post_cache_update May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_perform_op: + /cib/status/node_state[@id='2']: @in_ccm=false, @crm-debug-origin=post_cache_update May 10 15:25:13 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vmhost0-fsl.bcn/crmd/49, version=0.13.27) May 10 15:25:18 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_ping: Reporting our current digest to vmhost0-fsl.bcn: 06babcaef13cce59fbca2c77f46f92cd for 0.13.27 (0x56142f670c50 0) [11130] vmhost0-fsl.jsc.nasa.gov corosyncnotice [TOTEM ] The network interface is down. [11130] vmhost0-fsl.jsc.nasa.gov corosyncnotice [TOTEM ] adding new UDPU member {192.168.1.140} [11130] vmhost0-fsl.jsc.nasa.gov corosyncnotice [TOTEM ] adding new UDPU member {192.168.1.141} May 10 15:25:24 [11140] vmhost0-fsl.jsc.nasa.gov stonith-ng: notice: log_operation: Operation 'reboot' [11396] (call 2 from stonith_admin.11395) for host 'vmhost1-fsl.bcn' with device 'fence_vmhost1_ipmi' returned: 0 (OK) May 10 15:25:24 [11141] vmhost0-fsl.jsc.nasa.gov lrmd: info: log_finished: finished - rsc:fence_vmhost1_ipmi action:start call_id:12 exit-code:0 exec-time:11302ms queue-time:0ms May 10 15:25:24 [11144] vmhost0-fsl.jsc.nasa.gov crmd: notice: process_lrm_event: Result of start operation for fence_vmhost1_ipmi on vmhost0-fsl.bcn: 0 (ok) | call=12 key=fence_vmhost1_ipmi_start_0 confirmed=true cib-update=50 May 10 15:25:24 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Forwarding cib_modify operation for section status to all (origin=local/crmd/50) [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. May 10 15:25:54 [11144] vmhost0-fsl.jsc.nasa.gov crmd: warning: cib_rsc_callback: Resource update 50 failed: (rc=-62) Timer expired [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. May 10 15:26:33 [11144] vmhost0-fsl.jsc.nasa.gov crmd: warning: action_timer_callback: Timer popped (timeout=20000, abort_level=0, complete=false) May 10 15:26:33 [11144] vmhost0-fsl.jsc.nasa.gov crmd: error: print_synapse: [Action 7]: In-flight rsc op fence_vmhost1_ipmi_start_0 on vmhost0-fsl.bcn (priority: 0, waiting: none) May 10 15:26:33 [11144] vmhost0-fsl.jsc.nasa.gov crmd: notice: abort_transition_graph: Transition aborted: Action lost | source=action_timer_callback:911 complete=false May 10 15:26:33 [11144] vmhost0-fsl.jsc.nasa.gov crmd: warning: cib_action_update: rsc_op 7: fence_vmhost1_ipmi_start_0 on vmhost0-fsl.bcn timed out May 10 15:26:33 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: create_operation_update: cib_action_update: Updating resource fence_vmhost1_ipmi after start op Timed Out (interval=0) May 10 15:26:33 [11144] vmhost0-fsl.jsc.nasa.gov crmd: notice: run_graph: Transition 1 (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=1, Source=/var/lib/pacemaker/pengine/pe-input-24.bz2): Complete May 10 15:26:33 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd May 10 15:26:33 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Forwarding cib_modify operation for section status to all (origin=local/crmd/51) [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. May 10 15:27:04 [11144] vmhost0-fsl.jsc.nasa.gov crmd: error: cib_action_updated: Update 51 FAILED: Timer expired May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: notice: unpack_config: On loss of CCM Quorum: Ignore May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: determine_online_status_fencing: Node vmhost0-fsl.bcn is active May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: determine_online_status: Node vmhost0-fsl.bcn is online May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: unpack_node_loop: Node 1 is already processed May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: unpack_node_loop: Node 2 is already processed May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: unpack_node_loop: Node 1 is already processed May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: unpack_node_loop: Node 2 is already processed May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: common_print: fence_vmhost0_ipmi (stonith:fence_ipmilan): Started vmhost0-fsl.bcn May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: common_print: fence_vmhost1_ipmi (stonith:fence_ipmilan): Started vmhost0-fsl.bcn May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: RecurringOp: Start recurring monitor (60s) for fence_vmhost1_ipmi on vmhost0-fsl.bcn May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: LogActions: Leave fence_vmhost0_ipmi (Started vmhost0-fsl.bcn) May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: info: LogActions: Leave fence_vmhost1_ipmi (Started vmhost0-fsl.bcn) May 10 15:27:04 [11143] vmhost0-fsl.jsc.nasa.gov pengine: notice: process_pe_message: Calculated transition 2, saving inputs in /var/lib/pacemaker/pengine/pe-input-25.bz2 May 10 15:27:04 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE | input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response May 10 15:27:04 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: do_te_invoke: Processing graph 2 (ref=pe_calc-dc-1525984024-27) derived from /var/lib/pacemaker/pengine/pe-input-25.bz2 May 10 15:27:04 [11144] vmhost0-fsl.jsc.nasa.gov crmd: notice: te_rsc_command: Initiating start operation fence_vmhost1_ipmi_start_0 locally on vmhost0-fsl.bcn | action 5 May 10 15:27:04 [11144] vmhost0-fsl.jsc.nasa.gov crmd: info: do_lrm_rsc_op: Performing key=5:2:0:15826900-16df-4cd4-9f7d-7213b59d15a6 op=fence_vmhost1_ipmi_start_0 May 10 15:27:04 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Forwarding cib_modify operation for section status to all (origin=local/crmd/83) May 10 15:27:04 [11141] vmhost0-fsl.jsc.nasa.gov lrmd: info: log_execute: executing - rsc:fence_vmhost1_ipmi action:start call_id:13 May 10 15:27:04 [11141] vmhost0-fsl.jsc.nasa.gov lrmd: info: log_finished: finished - rsc:fence_vmhost1_ipmi action:start call_id:13 exit-code:0 exec-time:112ms queue-time:0ms May 10 15:27:04 [11144] vmhost0-fsl.jsc.nasa.gov crmd: notice: process_lrm_event: Result of start operation for fence_vmhost1_ipmi on vmhost0-fsl.bcn: 0 (ok) | call=13 key=fence_vmhost1_ipmi_start_0 confirmed=true cib-update=84 May 10 15:27:04 [11139] vmhost0-fsl.jsc.nasa.gov cib: info: cib_process_request: Forwarding cib_modify operation for section status to all (origin=local/crmd/84) [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. May 10 15:27:34 [11144] vmhost0-fsl.jsc.nasa.gov crmd: warning: cib_rsc_callback: Resource update 83 failed: (rc=-62) Timer expired May 10 15:27:34 [11144] vmhost0-fsl.jsc.nasa.gov crmd: warning: cib_rsc_callback: Resource update 84 failed: (rc=-62) Timer expired [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly. [11130] vmhost0-fsl.jsc.nasa.gov corosyncwarning [MAIN ] Totem is unable to form a cluster because of an operating system or network fault. The most common cause of this message is that the local firewall is configured improperly.