[ClusterLabs] Pacemaker fatal shutdown
Priyanka Balotra
priyanka.14balotra at gmail.com
Mon Jul 24 01:26:19 EDT 2023
Gentle Reminder!
On Fri, Jul 21, 2023 at 10:06 AM Priyanka Balotra <
priyanka.14balotra at gmail.com> wrote:
>
> Hi All,
> Any updates on this issue?
>
> Regards
> Priyanka
>
> On Thu, 20 Jul 2023 at 12:43 PM, Priyanka Balotra <
> priyanka.14balotra at gmail.com> wrote:
>
>> What I mainly want to understand is that:
>> - why "fatal failure" is coming
>> - why does pacemaker not start on the node after a node boots followed
>> by "pacemaker fatal failure" .
>> - How can this be handled?
>>
>> Thanks
>> Priyanka
>>
>> On Thu, Jul 20, 2023 at 12:41 PM Priyanka Balotra <
>> priyanka.14balotra at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Here are FILE-6 logs:
>>>
>>> 65710:Jul 17 14:16:51.517 FILE-6 pacemaker-controld [19415]
>>> (throttle_mode) debug: Current load is 0.760000 across 10 core(s)
>>> 65711:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (throttle_update) debug: Node FILE-2 has negligible load and supports at
>>> most 20 jobs; new job limit 20
>>> 65712:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (handle_request) debug: The throttle changed. Trigger a graph.
>>> 65713:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x00020000 (new_actions)
>>> for controller set by s_crmd_fsa:198
>>> 65714:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_JOIN_REQUEST: [ state=S_INTEGRATION
>>> cause=C_HA_MESSAGE origin=route_message ]
>>> 65715:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00020000 (an_action)
>>> for controller cleared by do_fsa_action:108
>>> 65716:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (do_dc_join_filter_offer) debug: Accepting join-1 request from FILE-2 |
>>> ref=join_request-crmd-1689603392-8
>>> 65717:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__update_peer_expected) info: do_dc_join_filter_offer: Node
>>> FILE-2[2] - expected state is now member (was (null))
>>> 65718:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (do_dc_join_filter_offer) debug: 2 nodes currently integrated in join-1
>>> 65719:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (check_join_state) debug: join-1: Integration of 2 peers complete |
>>> state=S_INTEGRATION for=do_dc_join_filter_offer
>>> 65720:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x00040000 (new_actions)
>>> for controller set by s_crmd_fsa:198
>>> 65721:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_INTEGRATED: [ state=S_INTEGRATION
>>> cause=C_FSA_INTERNAL origin=check_join_state ]
>>> 65722:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (do_state_transition) info: State transition S_INTEGRATION ->
>>> S_FINALIZE_JOIN | input=I_INTEGRATED cause=C_FSA_INTERNAL
>>> origin=check_join_state
>>> 65723:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x00000020
>>> (A_INTEGRATE_TIMER_STOP) for controller set by do_state_transition:559
>>> 65724:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x00000040
>>> (A_FINALIZE_TIMER_START) for controller set by do_state_transition:563
>>> 65725:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x00000200
>>> (A_DC_TIMER_STOP) for controller set by do_state_transition:569
>>> 65726:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (do_state_transition) debug: All cluster nodes (2) responded to join
>>> offer
>>> 65727:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000200 (an_action)
>>> for controller cleared by do_fsa_action:108
>>> 65728:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000020 (an_action)
>>> for controller cleared by do_fsa_action:108
>>> 65729:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000040 (an_action)
>>> for controller cleared by do_fsa_action:108
>>> 65730:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (controld_start_timer) debug: Started Finalization Timer (inject
>>> I_ELECTION if pops after 1800000ms, source=119)
>>> 65731:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00040000 (an_action)
>>> for controller cleared by do_fsa_action:108
>>> 65732:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (do_dc_join_finalize) debug: Finalizing join-1 for 2 nodes (sync'ing
>>> from local CIB)
>>> 65733:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (do_dc_join_finalize) debug: Requested CIB version <generation_tuple
>>> crm_feature_set="3.11.0" validate-with="pacemaker-3.7" epoch="24"
>>> num_updates="72" admin_epoch="0" cib-last-written="Thu Jul 13 13:11:46
>>> 2023" update-origin="FILE-1" update-client="cibadmin" update-user="root"
>>> have-quorum="1" dc-uuid="6"/>
>>> 65734:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-6=integrated
>>> 65735:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-2=integrated
>>> 65736:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-3=confirmed
>>> 65737:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-1=none
>>> 65738:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-5=confirmed
>>> 65739:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-4=confirmed
>>> 65740:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000
>>> (fsa_data->actions) for controller set by s_crmd_fsa:193
>>> 65741:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000000
>>> (new_actions) for controller set by s_crmd_fsa:198
>>> 65742:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_WAIT_FOR_EVENT: [
>>> state=S_FINALIZE_JOIN cause=C_HA_MESSAGE origin=do_te_invoke ]
>>> 65743:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65744:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415] (do_log)
>>> info: Input I_WAIT_FOR_EVENT received in state S_FINALIZE_JOIN from
>>> do_te_invoke
>>> 65745:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415] (do_log)
>>> debug: do_log <create_request_adv origin="do_cl_join_query" t="crmd"
>>> version="3.11.0" subt="request" reference="join_announce-crmd-1689603376-2"
>>> crm_task="join_announce" crm_sys_to="dc" crm_sys_from="crmd" src="FILE-1"/>
>>> 65746:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65747:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (abort_transition_graph) info: Transition 0 aborted: Peer Halt |
>>> source=do_te_invoke:135 complete=false
>>> 65748:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (register_fsa_input_adv) debug: Stalling the FSA pending further input:
>>> source=do_te_invoke cause=C_HA_MESSAGE data=0x55c6194ed4c0 queue=0
>>> 65749:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Exiting the FSA: queue=1, fsa_actions=0x0,
>>> stalled=true
>>> 65750:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (fsa_dump_queue) debug: queue[0.72]: input I_WAIT_FOR_EVENT raised by
>>> do_te_invoke(0x55c619869580.1) (cause=C_HA_MESSAGE)
>>> 65751:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000
>>> (fsa_data->actions) for controller set by s_crmd_fsa:193
>>> 65752:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000000
>>> (new_actions) for controller set by s_crmd_fsa:198
>>> 65753:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_WAIT_FOR_EVENT: [
>>> state=S_FINALIZE_JOIN cause=C_HA_MESSAGE origin=do_te_invoke ]
>>> 65754:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65755:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415] (do_log)
>>> info: Input I_WAIT_FOR_EVENT received in state S_FINALIZE_JOIN from
>>> do_te_invoke
>>> 65756:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415] (do_log)
>>> debug: do_log <create_request_adv origin="do_cl_join_query" t="crmd"
>>> version="3.11.0" subt="request" reference="join_announce-crmd-1689603376-2"
>>> crm_task="join_announce" crm_sys_to="dc" crm_sys_from="crmd" src="FILE-1"/>
>>> 65757:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65758:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (abort_transition_graph) info: Transition 0 aborted: Peer Halt |
>>> source=do_te_invoke:135 complete=false
>>> 65759:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (register_fsa_input_adv) debug: Stalling the FSA pending further input:
>>> source=do_te_invoke cause=C_HA_MESSAGE data=0x55c619869580 queue=0
>>> 65760:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Exiting the FSA: queue=1, fsa_actions=0x0,
>>> stalled=true
>>> 65761:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (fsa_dump_queue) debug: queue[0.73]: input I_WAIT_FOR_EVENT raised by
>>> do_te_invoke(0x55c6194ed4c0.1) (cause=C_HA_MESSAGE)
>>> 65762:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__execute_graph) debug: Transition 0 (Complete=33, Pending=2,
>>> Fired=0, Skipped=0, Incomplete=24,
>>> Source=/var/lib/pacemaker/pengine/pe-warn-0.bz2): In progress
>>> 65764:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (check_join_state) debug: join-1: Still waiting on 2 integrated
>>> nodes | state=S_FINALIZE_JOIN for=finalize_sync_callback
>>> 65765:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-6=integrated
>>> 65766:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-2=integrated
>>> 65767:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-3=confirmed
>>> 65768:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-1=none
>>> 65769:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-5=confirmed
>>> 65770:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-4=confirmed
>>> 65771:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (finalize_sync_callback) debug: Notifying 2 nodes of join-1 results
>>> 65772:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (finalize_join_for) debug: Acknowledging join-1 request from FILE-6
>>> 65773:Jul 17 14:16:55.085 FILE-6 pacemaker-controld [19415]
>>> (finalize_join_for) debug: Acknowledging join-1 request from FILE-2
>>> 65776:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (handle_request) debug: Raising I_JOIN_RESULT: join-1
>>> 65777:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000
>>> (fsa_data->actions) for controller set by s_crmd_fsa:193
>>> 65778:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000000
>>> (new_actions) for controller set by s_crmd_fsa:198
>>> 65779:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_WAIT_FOR_EVENT: [
>>> state=S_FINALIZE_JOIN cause=C_HA_MESSAGE origin=do_te_invoke ]
>>> 65780:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65781:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415] (do_log)
>>> info: Input I_WAIT_FOR_EVENT received in state S_FINALIZE_JOIN from
>>> do_te_invoke
>>> 65782:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415] (do_log)
>>> debug: do_log <create_request_adv origin="do_cl_join_query" t="crmd"
>>> version="3.11.0" subt="request" reference="join_announce-crmd-1689603376-2"
>>> crm_task="join_announce" crm_sys_to="dc" crm_sys_from="crmd" src="FILE-1"/>
>>> 65783:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65784:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (abort_transition_graph) info: Transition 0 aborted: Peer Halt |
>>> source=do_te_invoke:135 complete=false
>>> 65785:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (register_fsa_input_adv) debug: Stalling the FSA pending further input:
>>> source=do_te_invoke cause=C_HA_MESSAGE data=0x55c6194ed4c0 queue=1
>>> 65786:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Exiting the FSA: queue=2, fsa_actions=0x0,
>>> stalled=true
>>> 65787:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (fsa_dump_queue) debug: queue[0.74]: input I_JOIN_RESULT raised by
>>> route_message(0x55c619861a90.1) (cause=C_HA_MESSAGE)
>>> 65788:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (fsa_dump_queue) debug: queue[1.75]: input I_WAIT_FOR_EVENT raised by
>>> do_te_invoke(0x55c61986ed80.1) (cause=C_HA_MESSAGE)
>>> 65789:Jul 17 14:16:55.093 FILE-6 pacemaker-controld [19415]
>>> (pcmk__execute_graph) debug: Transition 0 (Complete=33, Pending=2,
>>> Fired=0, Skipped=0, Incomplete=24,
>>> Source=/var/lib/pacemaker/pengine/pe-warn-0.bz2): In progress
>>> 65792:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x00880000 (new_actions)
>>> for controller set by s_crmd_fsa:198
>>> 65793:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_JOIN_RESULT: [ state=S_FINALIZE_JOIN
>>> cause=C_HA_MESSAGE origin=route_message ]
>>> 65794:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00800000 (an_action)
>>> for controller cleared by do_fsa_action:108
>>> 65795:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource stonith-sbd after monitor op complete (interval=0)
>>> 65796:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource FILE_Filesystem after monitor op complete (interval=0)
>>> 65797:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource Service_pfile after monitor op complete (interval=0)
>>> 65798:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource Service_Postgresql after monitor op complete (interval=0)
>>> 65799:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource Service_esm_primary after monitor op complete (interval=0)
>>> 65800:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource Service_Postgrest after monitor op complete (interval=0)
>>> 65801:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource IP_Floating after monitor op complete (interval=0)
>>> 65802:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource Shared_Cluster_Backup after monitor op complete (interval=0)
>>> 65803:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (do_cl_join_finalize_respond) debug: Confirming join-1: sending local
>>> operation history to FILE-6
>>> 65804:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00080000 (an_action)
>>> for controller cleared by do_fsa_action:108
>>> 65805:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (do_dc_join_ack) debug: Ignoring 'join_ack_nack' message from FILE-6
>>> while waiting for 'join_confirm'
>>> 65806:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000
>>> (fsa_data->actions) for controller set by s_crmd_fsa:193
>>> 65807:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000000
>>> (new_actions) for controller set by s_crmd_fsa:198
>>> 65808:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_WAIT_FOR_EVENT: [
>>> state=S_FINALIZE_JOIN cause=C_HA_MESSAGE origin=do_te_invoke ]
>>> 65809:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65810:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415] (do_log)
>>> info: Input I_WAIT_FOR_EVENT received in state S_FINALIZE_JOIN from
>>> do_te_invoke
>>> 65811:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415] (do_log)
>>> debug: do_log <create_request_adv origin="do_cl_join_query" t="crmd"
>>> version="3.11.0" subt="request" reference="join_announce-crmd-1689603376-2"
>>> crm_task="join_announce" crm_sys_to="dc" crm_sys_from="crmd" src="FILE-1"/>
>>> 65812:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65813:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (abort_transition_graph) info: Transition 0 aborted: Peer Halt |
>>> source=do_te_invoke:135 complete=false
>>> 65814:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (register_fsa_input_adv) debug: Stalling the FSA pending further input:
>>> source=do_te_invoke cause=C_HA_MESSAGE data=0x55c61986ed80 queue=1
>>> 65815:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Exiting the FSA: queue=2, fsa_actions=0x0,
>>> stalled=true
>>> 65816:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (fsa_dump_queue) debug: queue[0.76]: input I_JOIN_RESULT raised by
>>> route_message(0x55c619871630.1) (cause=C_HA_MESSAGE)
>>> 65817:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (fsa_dump_queue) debug: queue[1.77]: input I_WAIT_FOR_EVENT raised by
>>> do_te_invoke(0x55c619861a90.1) (cause=C_HA_MESSAGE)
>>> 65818:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__execute_graph) debug: Transition 0 (Complete=33, Pending=2,
>>> Fired=0, Skipped=0, Incomplete=24,
>>> Source=/var/lib/pacemaker/pengine/pe-warn-0.bz2): In progress
>>> 65821:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x00880000 (new_actions)
>>> for controller set by s_crmd_fsa:198
>>> 65822:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_JOIN_RESULT: [ state=S_FINALIZE_JOIN
>>> cause=C_HA_MESSAGE origin=route_message ]
>>> 65823:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00800000 (an_action)
>>> for controller cleared by do_fsa_action:108
>>> 65824:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00080000 (an_action)
>>> for controller cleared by do_fsa_action:108
>>> 65825:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (controld_delete_node_state) info: Deleting resource history for node
>>> FILE-2 (via CIB call 71) | xpath=//node_state[@uname='FILE-2']/lrm
>>> 65826:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (do_dc_join_ack) debug: Updating node history for FILE-2 from join-1
>>> confirmation (via CIB call 72)
>>> 65827:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000
>>> (fsa_data->actions) for controller set by s_crmd_fsa:193
>>> 65828:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000000
>>> (new_actions) for controller set by s_crmd_fsa:198
>>> 65829:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_WAIT_FOR_EVENT: [
>>> state=S_FINALIZE_JOIN cause=C_HA_MESSAGE origin=do_te_invoke ]
>>> 65830:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65831:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415] (do_log)
>>> info: Input I_WAIT_FOR_EVENT received in state S_FINALIZE_JOIN from
>>> do_te_invoke
>>> 65832:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415] (do_log)
>>> debug: do_log <create_request_adv origin="do_cl_join_query" t="crmd"
>>> version="3.11.0" subt="request" reference="join_announce-crmd-1689603376-2"
>>> crm_task="join_announce" crm_sys_to="dc" crm_sys_from="crmd" src="FILE-1"/>
>>> 65833:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65834:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (abort_transition_graph) info: Transition 0 aborted: Peer Halt |
>>> source=do_te_invoke:135 complete=false
>>> 65835:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (register_fsa_input_adv) debug: Stalling the FSA pending further input:
>>> source=do_te_invoke cause=C_HA_MESSAGE data=0x55c619861a90 queue=1
>>> 65836:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Exiting the FSA: queue=2, fsa_actions=0x0,
>>> stalled=true
>>> 65837:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (fsa_dump_queue) debug: queue[0.78]: input I_JOIN_RESULT raised by
>>> route_message(0x55c6198798d0.1) (cause=C_HA_MESSAGE)
>>> 65838:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (fsa_dump_queue) debug: queue[1.79]: input I_WAIT_FOR_EVENT raised by
>>> do_te_invoke(0x55c619871630.1) (cause=C_HA_MESSAGE)
>>> 65839:Jul 17 14:16:55.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__execute_graph) debug: Transition 0 (Complete=33, Pending=2,
>>> Fired=0, Skipped=0, Incomplete=24,
>>> Source=/var/lib/pacemaker/pengine/pe-warn-0.bz2): In progress
>>> 65851:Jul 17 14:16:55.109 FILE-6 pacemaker-controld [19415]
>>> (cib_delete_callback) debug: Deletion of resource history for node
>>> FILE-2 (via CIB call 71) succeeded
>>> 65861:Jul 17 14:16:55.109 FILE-6 pacemaker-controld [19415]
>>> (te_update_diff) debug: Processing (cib_modify) diff: 0.24.72 -> 0.24.73
>>> (S_FINALIZE_JOIN)
>>> 65862:Jul 17 14:16:55.109 FILE-6 pacemaker-controld [19415]
>>> (join_update_complete_callback) debug: join-1 node history update (via
>>> CIB call 72) complete
>>> 65863:Jul 17 14:16:55.109 FILE-6 pacemaker-controld [19415]
>>> (check_join_state) debug: join-1: Still waiting on 1 finalized node
>>> | state=S_FINALIZE_JOIN for=join_update_complete_callback
>>> 65864:Jul 17 14:16:55.109 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-6=finalized
>>> 65865:Jul 17 14:16:55.109 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-2=confirmed
>>> 65866:Jul 17 14:16:55.109 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-3=confirmed
>>> 65867:Jul 17 14:16:55.109 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-1=none
>>> 65868:Jul 17 14:16:55.109 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-5=confirmed
>>> 65869:Jul 17 14:16:55.109 FILE-6 pacemaker-controld [19415]
>>> (crmd_join_phase_log) debug: join-1: FILE-4=confirmed
>>> 65876:Jul 17 14:17:21.517 FILE-6 pacemaker-controld [19415]
>>> (throttle_cib_load) debug: cib load: 0.001000 (3 ticks in 30s)
>>> 65877:Jul 17 14:17:21.517 FILE-6 pacemaker-controld [19415]
>>> (throttle_mode) debug: Current load is 0.960000 across 10 core(s)
>>> 65878:Jul 17 14:17:51.517 FILE-6 pacemaker-controld [19415]
>>> (throttle_cib_load) debug: cib load: 0.000333 (1 ticks in 30s)
>>> 65879:Jul 17 14:17:51.517 FILE-6 pacemaker-controld [19415]
>>> (throttle_mode) debug: Current load is 0.580000 across 10 core(s)
>>> 65883:Jul 17 14:18:20.085 FILE-6 pacemaker-fenced [19411]
>>> (process_remote_stonith_exec) debug: Finalizing action 'reboot'
>>> targeting FILE-2 on behalf of pacemaker-controld.19415 at FILE-6: OK |
>>> rc=0 id=4e523b34
>>> 65884:Jul 17 14:18:20.085 FILE-6 pacemaker-fenced [19411]
>>> (remote_op_done) notice: Operation 'reboot' targeting FILE-2 by FILE-4
>>> for pacemaker-controld.19415 at FILE-6: OK | id=4e523b34
>>> 65886:Jul 17 14:18:20.085 FILE-6 pacemaker-controld [19415]
>>> (tengine_stonith_callback) notice: Stonith operation
>>> 3/63:0:0:232e6505-2e98-4a79-b6ce-5f26d9cba645: OK (0)
>>> 65887:Jul 17 14:18:20.085 FILE-6 pacemaker-controld [19415]
>>> (tengine_stonith_callback) info: Stonith operation 3 for FILE-2
>>> passed
>>> 65888:Jul 17 14:18:20.085 FILE-6 pacemaker-controld [19415]
>>> (pcmk__update_peer_expected) info: crmd_peer_down: Node FILE-2[2] -
>>> expected state is now down (was member)
>>> 65889:Jul 17 14:18:20.085 FILE-6 pacemaker-controld [19415]
>>> (send_stonith_update) debug: Sending fencing update 73 for FILE-2
>>> 65890:Jul 17 14:18:20.085 FILE-6 pacemaker-controld [19415]
>>> (controld_delete_node_state) info: Deleting all state for node FILE-2
>>> (via CIB call 74) | xpath=//node_state[@uname='FILE-2']/*
>>> 65892:Jul 17 14:18:20.089 FILE-6 pacemaker-controld [19415]
>>> (exec_alert_list) info: Sending fencing alert via pf-ha-alert to (null)
>>> 65896:Jul 17 14:18:20.089 FILE-6 pacemaker-controld [19415]
>>> (tengine_stonith_notify) notice: Peer FILE-2 was terminated (reboot) by
>>> FILE-4 on behalf of pacemaker-controld.19415: OK | initiator=FILE-6
>>> ref=4e523b34-dcb1-40bc-a296-5e984b4e6b00
>>> 65897:Jul 17 14:18:20.089 FILE-6 pacemaker-controld [19415]
>>> (send_stonith_update) debug: Sending fencing update 75 for FILE-2
>>> 65898:Jul 17 14:18:20.089 FILE-6 pacemaker-controld [19415]
>>> (controld_delete_node_state) info: Deleting all state for node FILE-2
>>> (via CIB call 76) | xpath=//node_state[@uname='FILE-2']/*
>>> 65899:Jul 17 14:18:20.089 FILE-6 pacemaker-controld [19415]
>>> (pcmk__execute_graph) debug: Transition 0 (Complete=34, Pending=1,
>>> Fired=0, Skipped=0, Incomplete=24,
>>> Source=/var/lib/pacemaker/pengine/pe-warn-0.bz2): In progress
>>> 65907:Jul 17 14:18:20.089 FILE-6 pacemaker-controld [19415]
>>> (te_update_diff) debug: Processing (cib_modify) diff: 0.24.73 -> 0.24.74
>>> (S_FINALIZE_JOIN)
>>> 65908:Jul 17 14:18:20.089 FILE-6 pacemaker-controld [19415]
>>> (cib_fencing_updated) info: Fencing update 73 for FILE-2: complete
>>> 65916:Jul 17 14:18:20.093 FILE-6 pacemaker-controld [19415]
>>> (te_update_diff) debug: Processing (cib_delete) diff: 0.24.74 -> 0.24.75
>>> (S_FINALIZE_JOIN)
>>> 65919:Jul 17 14:18:20.093 FILE-6 pacemaker-controld [19415]
>>> (match_down_event) debug: Shutdown action 63
>>> (stonith-FILE-2-reboot) found for node 2
>>> 65920:Jul 17 14:18:20.093 FILE-6 pacemaker-controld [19415]
>>> (cib_delete_callback) debug: Deletion of all state for node FILE-2
>>> (via CIB call 74) succeeded
>>> 65921:Jul 17 14:18:20.093 FILE-6 pacemaker-controld [19415]
>>> (cib_fencing_updated) info: Fencing update 75 for FILE-2: complete
>>> 65924:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (cib_delete_callback) debug: Deletion of all state for node FILE-2
>>> (via CIB call 76) succeeded
>>> 65927:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415] (node_left)
>>> info: Group crmd event 5: FILE-2 (node 2 pid 15962) left for unknown
>>> reason
>>> 65928:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (crm_update_peer_proc) info: node_left: Node FILE-2[2] - corosync-cpg
>>> is now offline
>>> 65929:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (peer_update_callback) info: Node FILE-2 is no longer a peer | DC=true
>>> old=0x4000000 new=0x0000000
>>> 65930:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (controld_delete_node_state) info: Deleting transient attributes for
>>> node FILE-2 (via CIB call 77) |
>>> xpath=//node_state[@uname='FILE-2']/transient_attributes
>>> 65932:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (match_down_event) debug: Shutdown action 63
>>> (stonith-FILE-2-reboot) found for node 2
>>> 65933:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk_cpg_membership) info: Group crmd event 5: FILE-3 (node 3 pid
>>> 19250) is member
>>> 65934:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk_cpg_membership) info: Group crmd event 5: FILE-4 (node 4 pid
>>> 19122) is member
>>> 65935:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk_cpg_membership) info: Group crmd event 5: FILE-5 (node 5 pid
>>> 19273) is member
>>> 65936:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk_cpg_membership) info: Group crmd event 5: FILE-6 (node 6 pid
>>> 19415) is member
>>> 65938:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x00880000 (new_actions)
>>> for controller set by s_crmd_fsa:198
>>> 65939:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_JOIN_RESULT: [ state=S_FINALIZE_JOIN
>>> cause=C_HA_MESSAGE origin=route_message ]
>>> 65940:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00800000 (an_action)
>>> for controller cleared by do_fsa_action:108
>>> 65941:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00080000 (an_action)
>>> for controller cleared by do_fsa_action:108
>>> 65942:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (controld_delete_node_state) info: Deleting resource history for node
>>> FILE-6 (via CIB call 79) | xpath=//node_state[@uname='FILE-6']/lrm
>>> 65943:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource stonith-sbd after monitor op complete (interval=0)
>>> 65945:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource FILE_Filesystem after monitor op complete (interval=0)
>>> 65946:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource Service_pfile after monitor op complete (interval=0)
>>> 65947:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource Service_Postgresql after monitor op complete (interval=0)
>>> 65948:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource Service_esm_primary after monitor op complete (interval=0)
>>> 65949:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource Service_Postgrest after monitor op complete (interval=0)
>>> 65950:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource IP_Floating after monitor op complete (interval=0)
>>> 65951:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__create_history_xml) debug: build_active_RAs: Updating
>>> resource Shared_Cluster_Backup after monitor op complete (interval=0)
>>> 65952:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (do_dc_join_ack) debug: Updating local node history for join-1 from query
>>> result (via CIB call 80)
>>> 65954:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000
>>> (fsa_data->actions) for controller set by s_crmd_fsa:193
>>> 65955:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000000
>>> (new_actions) for controller set by s_crmd_fsa:198
>>> 65956:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_WAIT_FOR_EVENT: [
>>> state=S_FINALIZE_JOIN cause=C_HA_MESSAGE origin=do_te_invoke ]
>>> 65957:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65958:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415] (do_log)
>>> info: Input I_WAIT_FOR_EVENT received in state S_FINALIZE_JOIN from
>>> do_te_invoke
>>> 65959:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415] (do_log)
>>> debug: do_log <create_request_adv origin="do_cl_join_query" t="crmd"
>>> version="3.11.0" subt="request" reference="join_announce-crmd-1689603376-2"
>>> crm_task="join_announce" crm_sys_to="dc" crm_sys_from="crmd" src="FILE-1"/>
>>> 65960:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65961:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (abort_transition_graph) info: Transition 0 aborted: Peer Halt |
>>> source=do_te_invoke:135 complete=false
>>> 65962:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (register_fsa_input_adv) debug: Stalling the FSA pending further input:
>>> source=do_te_invoke cause=C_HA_MESSAGE data=0x55c619871630 queue=0
>>> 65963:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Exiting the FSA: queue=1, fsa_actions=0x0,
>>> stalled=true
>>> 65964:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (fsa_dump_queue) debug: queue[0.80]: input I_WAIT_FOR_EVENT raised by
>>> do_te_invoke(0x55c6198798d0.1) (cause=C_HA_MESSAGE)
>>> 65966:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__execute_graph) debug: Transition 0 (Complete=34, Pending=1,
>>> Fired=0, Skipped=0, Incomplete=24,
>>> Source=/var/lib/pacemaker/pengine/pe-warn-0.bz2): In progress
>>> 65967:Jul 17 14:18:20.097 FILE-6 pacemaker-fenced [19411]
>>> (process_remote_stonith_exec) debug: Finalizing action 'reboot'
>>> targeting FILE-1 on behalf of pacemaker-controld.19415 at FILE-6: OK |
>>> rc=0 id=446afc42
>>> 65968:Jul 17 14:18:20.097 FILE-6 pacemaker-fenced [19411]
>>> (remote_op_done) notice: Operation 'reboot' targeting FILE-1 by FILE-5
>>> for pacemaker-controld.19415 at FILE-6: OK | id=446afc42
>>> 65970:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (tengine_stonith_callback) notice: Stonith operation
>>> 4/62:0:0:232e6505-2e98-4a79-b6ce-5f26d9cba645: OK (0)
>>> 65971:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (tengine_stonith_callback) info: Stonith operation 4 for FILE-1
>>> passed
>>> 65972:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (pcmk__update_peer_expected) info: crmd_peer_down: Node FILE-1[1] -
>>> expected state is now down (was pending)
>>> 65973:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (send_stonith_update) debug: Sending fencing update 81 for FILE-1
>>> 65974:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (controld_delete_node_state) info: Deleting all state for node FILE-1
>>> (via CIB call 82) | xpath=//node_state[@uname='FILE-1']/*
>>> 65975:Jul 17 14:18:20.097 FILE-6 pacemaker-controld [19415]
>>> (exec_alert_list) info: Sending fencing alert via pf-ha-alert to (null)
>>> 65979:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (tengine_stonith_notify) notice: Peer FILE-1 was terminated (reboot) by
>>> FILE-5 on behalf of pacemaker-controld.19415: OK | initiator=FILE-6
>>> ref=446afc42-b46e-47af-9fac-0fa87c1c5e57
>>> 65980:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (send_stonith_update) debug: Sending fencing update 83 for FILE-1
>>> 65982:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (controld_delete_node_state) info: Deleting all state for node FILE-1
>>> (via CIB call 84) | xpath=//node_state[@uname='FILE-1']/*
>>> 65983:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (cib_delete_callback) debug: Deletion of transient attributes for node
>>> FILE-2 (via CIB call 77) succeeded
>>> 65984:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (pcmk__execute_graph) notice: Transition 0 (Complete=35, Pending=0,
>>> Fired=0, Skipped=3, Incomplete=24,
>>> Source=/var/lib/pacemaker/pengine/pe-warn-0.bz2): Stopped
>>> 65985:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (te_graph_trigger) debug: Transition 0 is now complete
>>> 65986:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (notify_crmd) debug: Processing transition completion in state
>>> S_FINALIZE_JOIN
>>> 65987:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (notify_crmd) debug: Transition 0 status: restart - Node join
>>> 65988:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000
>>> (fsa_data->actions) for controller set by s_crmd_fsa:193
>>> 65989:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000000
>>> (new_actions) for controller set by s_crmd_fsa:198
>>> 65990:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_WAIT_FOR_EVENT: [
>>> state=S_FINALIZE_JOIN cause=C_HA_MESSAGE origin=do_te_invoke ]
>>> 65991:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65992:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415] (do_log)
>>> info: Input I_WAIT_FOR_EVENT received in state S_FINALIZE_JOIN from
>>> do_te_invoke
>>> 65993:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415] (do_log)
>>> debug: do_log <create_request_adv origin="do_cl_join_query" t="crmd"
>>> version="3.11.0" subt="request" reference="join_announce-crmd-1689603376-2"
>>> crm_task="join_announce" crm_sys_to="dc" crm_sys_from="crmd" src="FILE-1"/>
>>> 65994:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000
>>> (an_action) for controller cleared by do_fsa_action:108
>>> 65995:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (abort_transition_graph) info: Transition 0 aborted: Peer Halt |
>>> source=do_te_invoke:135 complete=true
>>> 65996:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (s_crmd_fsa) debug: Processing I_PE_CALC: [ state=S_FINALIZE_JOIN
>>> cause=C_FSA_INTERNAL origin=abort_transition_graph ]
>>> 66024:Jul 17 14:18:20.101 FILE-6 pacemaker-controld [19415]
>>> (cib_delete_callback) debug: Deletion of resource history for node
>>> FILE-6 (via CIB call 79) succeeded
>>> 66063:Jul 17 14:18:20.105 FILE-6 pacemaker-controld [19415]
>>> (join_update_complete_callback) debug: join-1 node history update (via
>>> CIB call 80) complete
>>> 66064:Jul 17 14:18:20.105 FILE-6 pacemaker-controld [19415]
>>> (check_join_state) debug: join-1: Complete | state=S_FINALIZE_JOIN
>>> for=join_update_complete_callback
>>> 66068:Jul 17 14:18:20.105 FILE-6 pacemaker-controld [19415]
>>> (pcmk__set_flags_as) debug: FSA action flags 0x800400000000
>>> (new_actions) for controller set by s_crmd_fsa:198
>>>
>>> Thanks
>>> Priyanka
>>>
>>> On Thu, Jul 20, 2023 at 11:53 AM Reid Wahl <nwahl at redhat.com> wrote:
>>>
>>>> On Wed, Jul 19, 2023 at 8:33 PM Priyanka Balotra
>>>> <priyanka.14balotra at gmail.com> wrote:
>>>> >
>>>> > Sure,
>>>> > Here are the logs:
>>>> >
>>>> >
>>>> > 63138:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (post_cache_update) debug: Updated cache after membership event 44.
>>>> > 63139:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x200000000
>>>> (A_ELECTION_CHECK) for controller set by post_cache_update:81
>>>> > 63140:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000002 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63141:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (do_started) info: Delaying start, Config not read (0000000000000040)
>>>> > 63142:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (register_fsa_input_adv) debug: Stalling the FSA pending further input:
>>>> source=do_started cause=C_FSA_INTERNAL data=(nil) queue=0
>>>> > 63143:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x00000002
>>>> (with_actions) for controller set by register_fsa_input_adv:88
>>>> > 63144:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (s_crmd_fsa) debug: Exiting the FSA: queue=0,
>>>> fsa_actions=0x200000002, stalled=true
>>>> > 63145:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (config_query_callback) debug: Call 3 : Parsing CIB options
>>>> > 63146:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (config_query_callback) debug: Shutdown escalation occurs if DC has not
>>>> responded to request in 1200000ms
>>>> > 63147:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (config_query_callback) debug: Re-run scheduler after 900000ms of
>>>> inactivity
>>>> > 63148:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pe_unpack_alerts) debug: Alert pf-ha-alert:
>>>> path=/usr/lib/ocf/resource.d/pacemaker/pf_ha_alert.sh timeout=30000ms
>>>> tstamp-format='%H:%M:%S.%06N' 0 vars
>>>> > 63149:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000002 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63150:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (do_started) debug: Init server comms
>>>> > 63151:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcs_us_publish) info: server name: crmd
>>>> > 63152:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (do_started) notice: Pacemaker controller successfully started and
>>>> accepting connections
>>>> > 63153:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x200000000 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63154:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (do_election_check) debug: Ignoring election check because we are
>>>> not in an election
>>>> > 63155:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000100100
>>>> (new_actions) for controller set by s_crmd_fsa:198
>>>> > 63156:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (s_crmd_fsa) debug: Processing I_PENDING: [ state=S_STARTING
>>>> cause=C_FSA_INTERNAL origin=do_started ]
>>>> > 63157:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000000
>>>> (an_action) for controller cleared by do_fsa_action:108
>>>> > 63158:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (do_log) info: Input I_PENDING received in state S_STARTING from
>>>> do_started
>>>> > 63159:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (do_state_transition) notice: State transition S_STARTING -> S_PENDING
>>>> | input=I_PENDING cause=C_FSA_INTERNAL origin=do_started
>>>> > 63160:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x00000020
>>>> (A_INTEGRATE_TIMER_STOP) for controller set by do_state_transition:559
>>>> > 63161:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x00000080
>>>> (A_FINALIZE_TIMER_STOP) for controller set by do_state_transition:565
>>>> > 63162:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000020 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63163:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000080 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63164:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00100000 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63165:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962]
>>>> (do_cl_join_query) debug: Querying for a DC
>>>> > 63166:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000100 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63167:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962]
>>>> (controld_start_timer) debug: Started Election Trigger (inject
>>>> I_DC_TIMEOUT if pops after 20000ms, source=18)
>>>> > 63168:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962]
>>>> (stonith_api_signon) debug: Attempting fencer connection by
>>>> pacemaker-controld with mainloop
>>>> > 63175:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:131085; real_size:135168;
>>>> rb->word_size:33792
>>>> > 63176:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:131085; real_size:135168;
>>>> rb->word_size:33792
>>>> > 63177:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:131085; real_size:135168;
>>>> rb->word_size:33792
>>>> > 63178:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958]
>>>> (stonith_command) debug: Processing register 8 from client
>>>> pacemaker-controld.15962 with call options 0x00000000
>>>> > 63179:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958]
>>>> (stonith_command) debug: Processed register from client
>>>> pacemaker-controld.15962: OK (rc=0)
>>>> > 63180:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962]
>>>> (stonith_api_signon) debug: Connection to fencer by
>>>> pacemaker-controld succeeded (registration token:
>>>> 5552b1b4-f725-46ac-b239-e404cadd8d94)
>>>> > 63181:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958]
>>>> (stonith_command) debug: Processing st_notify 9 from client
>>>> pacemaker-controld.15962 with call options 0x00000000
>>>> > 63182:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958]
>>>> (handle_request) debug: Enabling st_notify_disconnect callbacks for
>>>> client pacemaker-controld.15962
>>>> > 63183:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958]
>>>> (stonith_command) debug: Processed st_notify from client
>>>> pacemaker-controld.15962: OK (rc=0)
>>>> > 63184:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958]
>>>> (stonith_command) debug: Processing st_notify 10 from client
>>>> pacemaker-controld.15962 with call options 0x00000000
>>>> > 63185:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958]
>>>> (handle_request) debug: Enabling st_notify_fence callbacks for client
>>>> pacemaker-controld.15962
>>>> > 63186:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958]
>>>> (stonith_command) debug: Processed st_notify from client
>>>> pacemaker-controld.15962: OK (rc=0)
>>>> > 63187:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958]
>>>> (stonith_command) debug: Processing st_notify 11 from client
>>>> pacemaker-controld.15962 with call options 0x00000000
>>>> > 63188:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958]
>>>> (handle_request) debug: Enabling st_notify_history_synced callbacks for
>>>> client pacemaker-controld.15962
>>>> > 63189:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958]
>>>> (stonith_command) debug: Processed st_notify from client
>>>> pacemaker-controld.15962: OK (rc=0)
>>>> > 63190:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962]
>>>> (te_trigger_stonith_history_sync) info: Fence history will be synchronized
>>>> cluster-wide within 30 seconds
>>>> > 63191:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962]
>>>> (te_connect_stonith) notice: Fencer successfully connected
>>>> > 63192:Jul 17 14:16:32.664 FILE-2 pacemaker-controld [15962]
>>>> (quorum_notification_cb) info: Quorum retained | membership=48 members=5
>>>> > 63193:Jul 17 14:16:32.664 FILE-2 pacemaker-controld [15962]
>>>> (quorum_notification_cb) debug: Member[0] 2
>>>> > 63194:Jul 17 14:16:32.664 FILE-2 pacemaker-controld [15962]
>>>> (quorum_notification_cb) debug: Member[1] 4
>>>> > 63195:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63196:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63197:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63198:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63199:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-e4qK7U/qb-request-cmap-header
>>>> > 63200:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-e4qK7U/qb-response-cmap-header
>>>> > 63201:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-e4qK7U/qb-event-cmap-header
>>>> > 63202:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 4
>>>> > 63203:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 4
>>>> > 63204:Jul 17 14:16:32.672 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63205:Jul 17 14:16:32.672 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63206:Jul 17 14:16:32.672 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63209:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63210:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-YYxILU/qb-request-cmap-header
>>>> > 63211:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-YYxILU/qb-response-cmap-header
>>>> > 63212:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-YYxILU/qb-event-cmap-header
>>>> > 63213:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 4
>>>> > 63214:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962]
>>>> (quorum_notification_cb) info: Obtaining name for new node 4
>>>> > 63218:Jul 17 14:16:32.684 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63222:Jul 17 14:16:32.684 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63225:Jul 17 14:16:32.684 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63240:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63241:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-Cy8QVV/qb-request-cmap-header
>>>> > 63242:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-Cy8QVV/qb-response-cmap-header
>>>> > 63243:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-Cy8QVV/qb-event-cmap-header
>>>> > 63244:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 4
>>>> > 63245:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 4
>>>> > 63246:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962]
>>>> (quorum_notification_cb) debug: Member[2] 3
>>>> > 63259:Jul 17 14:16:32.700 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63265:Jul 17 14:16:32.700 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63267:Jul 17 14:16:32.700 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63298:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63299:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-34-0DHKhX/qb-request-cmap-header
>>>> > 63300:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-34-0DHKhX/qb-response-cmap-header
>>>> > 63301:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-34-0DHKhX/qb-event-cmap-header
>>>> > 63302:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 3
>>>> > 63303:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 3
>>>> > 63307:Jul 17 14:16:32.720 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63313:Jul 17 14:16:32.720 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63320:Jul 17 14:16:32.720 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63351:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63352:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-34-V0bQlV/qb-request-cmap-header
>>>> > 63353:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-34-V0bQlV/qb-response-cmap-header
>>>> > 63355:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-34-V0bQlV/qb-event-cmap-header
>>>> > 63356:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 3
>>>> > 63357:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962]
>>>> (quorum_notification_cb) info: Obtaining name for new node 3
>>>> > 63365:Jul 17 14:16:32.736 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63372:Jul 17 14:16:32.736 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63374:Jul 17 14:16:32.736 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63415:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63416:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-34-EAFzTX/qb-request-cmap-header
>>>> > 63417:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-34-EAFzTX/qb-response-cmap-header
>>>> > 63418:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-34-EAFzTX/qb-event-cmap-header
>>>> > 63419:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 3
>>>> > 63420:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 3
>>>> > 63421:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962]
>>>> (quorum_notification_cb) debug: Member[3] 6
>>>> > 63425:Jul 17 14:16:32.752 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63426:Jul 17 14:16:32.752 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63427:Jul 17 14:16:32.752 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63479:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63480:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-33-q3mFYU/qb-request-cmap-header
>>>> > 63481:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-33-q3mFYU/qb-response-cmap-header
>>>> > 63482:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-33-q3mFYU/qb-event-cmap-header
>>>> > 63483:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 6
>>>> > 63484:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 6
>>>> > 63485:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63486:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63487:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63490:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63491:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-EcEbfV/qb-request-cmap-header
>>>> > 63492:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-EcEbfV/qb-response-cmap-header
>>>> > 63493:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-EcEbfV/qb-event-cmap-header
>>>> > 63494:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 6
>>>> > 63495:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962]
>>>> (quorum_notification_cb) info: Obtaining name for new node 6
>>>> > 63499:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63502:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63505:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63508:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63509:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-fLk4xW/qb-request-cmap-header
>>>> > 63510:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-fLk4xW/qb-response-cmap-header
>>>> > 63511:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-fLk4xW/qb-event-cmap-header
>>>> > 63512:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 6
>>>> > 63513:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 6
>>>> > 63514:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962]
>>>> (quorum_notification_cb) debug: Member[4] 5
>>>> > 63517:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63518:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63521:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63528:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63529:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-ushXmW/qb-request-cmap-header
>>>> > 63530:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-ushXmW/qb-response-cmap-header
>>>> > 63531:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-ushXmW/qb-event-cmap-header
>>>> > 63532:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 5
>>>> > 63533:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 5
>>>> > 63534:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63535:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63536:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63537:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63538:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-x3qVkW/qb-request-cmap-header
>>>> > 63539:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-x3qVkW/qb-response-cmap-header
>>>> > 63540:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-x3qVkW/qb-event-cmap-header
>>>> > 63541:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 5
>>>> > 63542:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962]
>>>> (quorum_notification_cb) info: Obtaining name for new node 5
>>>> > 63543:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63544:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63545:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63546:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63547:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-gUNSFU/qb-request-cmap-header
>>>> > 63548:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-gUNSFU/qb-response-cmap-header
>>>> > 63549:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-gUNSFU/qb-event-cmap-header
>>>> > 63550:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 5
>>>> > 63551:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 5
>>>> > 63552:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (update_peer_state_iter) notice: Node (null) state is now lost | nodeid=1
>>>> previous=member source=pcmk__reap_unseen_nodes
>>>> > 63553:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (post_cache_update) debug: Updated cache after membership event 48.
>>>> > 63554:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x200000000
>>>> (A_ELECTION_CHECK) for controller set by post_cache_update:81
>>>> > 63555:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x200000000 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63556:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (do_election_check) debug: Ignoring election check because we are
>>>> not in an election
>>>> > 63557:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (pcmk_cpg_membership) info: Group crmd event 0: node 2 pid 15962
>>>> joined via cpg_join
>>>> > 63558:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962]
>>>> (pcmk_cpg_membership) info: Group crmd event 0: FILE-2 (node 2 pid
>>>> 15962) is member
>>>> > 63559:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63560:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63561:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63564:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63565:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-5PH1gV/qb-request-cmap-header
>>>> > 63566:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-5PH1gV/qb-response-cmap-header
>>>> > 63567:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-5PH1gV/qb-event-cmap-header
>>>> > 63568:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 3
>>>> > 63569:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 3
>>>> > 63570:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (pcmk_cpg_membership) info: Group crmd event 0: peer node (node 3 pid
>>>> 19250) is member
>>>> > 63571:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (crm_update_peer_proc) info: pcmk_cpg_membership: Node (null)[3] -
>>>> corosync-cpg is now online
>>>> > 63572:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962]
>>>> (peer_update_callback) debug: Sending hello to node 3 so that it learns
>>>> our node name
>>>> > 63573:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63574:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63575:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63576:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63577:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-QATDEV/qb-request-cmap-header
>>>> > 63578:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-QATDEV/qb-response-cmap-header
>>>> > 63579:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-QATDEV/qb-event-cmap-header
>>>> > 63580:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 4
>>>> > 63581:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 4
>>>> > 63582:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (pcmk_cpg_membership) info: Group crmd event 0: peer node (node 4 pid
>>>> 19122) is member
>>>> > 63583:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (crm_update_peer_proc) info: pcmk_cpg_membership: Node (null)[4] -
>>>> corosync-cpg is now online
>>>> > 63584:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962]
>>>> (peer_update_callback) debug: Sending hello to node 4 so that it learns
>>>> our node name
>>>> > 63585:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63586:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63587:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63588:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63589:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-TVzR1T/qb-request-cmap-header
>>>> > 63590:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-TVzR1T/qb-response-cmap-header
>>>> > 63591:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-TVzR1T/qb-event-cmap-header
>>>> > 63592:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 5
>>>> > 63593:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 5
>>>> > 63594:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (pcmk_cpg_membership) info: Group crmd event 0: peer node (node 5 pid
>>>> 19273) is member
>>>> > 63595:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (crm_update_peer_proc) info: pcmk_cpg_membership: Node (null)[5] -
>>>> corosync-cpg is now online
>>>> > 63596:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962]
>>>> (peer_update_callback) debug: Sending hello to node 5 so that it learns
>>>> our node name
>>>> > 63597:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63598:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63599:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_open_2) debug: shm size:1048589; real_size:1052672;
>>>> rb->word_size:263168
>>>> > 63600:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()
>>>> > 63601:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-8LRaoV/qb-request-cmap-header
>>>> > 63602:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-8LRaoV/qb-response-cmap-header
>>>> > 63603:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (qb_rb_close_helper) debug: Closing ringbuffer:
>>>> /dev/shm/qb-13142-15962-31-8LRaoV/qb-event-cmap-header
>>>> > 63604:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__corosync_name) info: Unable to get node name for nodeid 6
>>>> > 63605:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (get_node_name) notice: Could not obtain a node name for corosync node
>>>> with id 6
>>>> > 63606:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (pcmk_cpg_membership) info: Group crmd event 0: peer node (node 6 pid
>>>> 19415) is member
>>>> > 63607:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (crm_update_peer_proc) info: pcmk_cpg_membership: Node (null)[6] -
>>>> corosync-cpg is now online
>>>> > 63608:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (peer_update_callback) debug: Sending hello to node 6 so that it learns
>>>> our node name
>>>> > 63609:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (get_xpath_object) debug: No match for //st_notify_history_synced
>>>> in /notify
>>>> > 63610:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (stonith_api_del_notification) debug: Removing callback for
>>>> st_notify_history_synced events
>>>> > 63611:Jul 17 14:16:32.792 FILE-2 pacemaker-fenced [15958]
>>>> (stonith_command) debug: Processing st_notify 12 from client
>>>> pacemaker-controld.15962 with call options 0x00000000
>>>> > 63612:Jul 17 14:16:32.792 FILE-2 pacemaker-fenced [15958]
>>>> (handle_request) debug: Disabling st_notify_history_synced callbacks for
>>>> client pacemaker-controld.15962
>>>> > 63613:Jul 17 14:16:32.792 FILE-2 pacemaker-fenced [15958]
>>>> (stonith_command) debug: Processed st_notify from client
>>>> pacemaker-controld.15962: OK (rc=0)
>>>> > 63614:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (tengine_stonith_history_synced) debug: Fence-history synced - cancel all
>>>> timers
>>>> > 63615:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (crm_get_peer) info: Node 4 is now known as FILE-4
>>>> > 63616:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962]
>>>> (update_peer_uname) warning: Node names with capitals are
>>>> discouraged, consider changing 'FILE-4'
>>>> > 63617:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962]
>>>> (peer_update_callback) info: Cluster node FILE-4 is now member
>>>> > 63618:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962]
>>>> (crm_get_peer) info: Node 3 is now known as FILE-3
>>>> > 63619:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962]
>>>> (update_peer_uname) warning: Node names with capitals are
>>>> discouraged, consider changing 'FILE-3'
>>>> > 63620:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962]
>>>> (peer_update_callback) info: Cluster node FILE-3 is now member
>>>> > 63621:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962]
>>>> (crm_get_peer) info: Node 5 is now known as FILE-5
>>>> > 63622:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962]
>>>> (update_peer_uname) warning: Node names with capitals are
>>>> discouraged, consider changing 'FILE-5'
>>>> > 63623:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962]
>>>> (peer_update_callback) info: Cluster node FILE-5 is now member
>>>> > 63640:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (crm_get_peer) info: Node 6 is now known as FILE-6
>>>> > 63641:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (update_peer_uname) warning: Node names with capitals are
>>>> discouraged, consider changing 'FILE-6'
>>>> > 63642:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (peer_update_callback) info: Cluster node FILE-6 is now member
>>>> > 63643:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (handle_request) debug: Raising I_JOIN_OFFER: join-1
>>>> > 63644:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x00400200 (new_actions)
>>>> for controller set by s_crmd_fsa:198
>>>> > 63645:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (s_crmd_fsa) debug: Processing I_JOIN_OFFER: [ state=S_PENDING
>>>> cause=C_HA_MESSAGE origin=route_message ]
>>>> > 63646:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000200 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63647:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00400000 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63648:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (update_dc) info: Set DC to FILE-6 (3.11.0)
>>>> > 63649:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__update_peer_expected) info: update_dc: Node FILE-6[6] -
>>>> expected state is now member (was (null))
>>>> > 63650:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x00000200
>>>> (A_DC_TIMER_STOP) for controller set by do_cl_join_offer_respond:147
>>>> > 63651:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000200 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63788:Jul 17 14:16:32.884 FILE-2 pacemaker-controld [15962]
>>>> (do_cib_replaced) debug: Updating the CIB after a replace: DC=false
>>>> > 63811:Jul 17 14:16:32.892 FILE-2 pacemaker-controld [15962]
>>>> (join_query_callback) debug: Respond to join offer join-1 from FILE-6
>>>> > 63819:Jul 17 14:16:55.080 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__procfs_pid_of) info: Found pacemaker-based active as process
>>>> 15957
>>>> > 63820:Jul 17 14:16:55.080 FILE-2 pacemaker-controld [15962]
>>>> (throttle_cib_load) debug: Init 6 + 2 ticks at 1689603415 (100 tps)
>>>> > 63821:Jul 17 14:16:55.080 FILE-2 pacemaker-controld [15962]
>>>> (throttle_mode) debug: Current load is 0.980000 across 10 core(s)
>>>> > 63822:Jul 17 14:16:55.080 FILE-2 pacemaker-controld [15962]
>>>> (throttle_send_command) info: New throttle mode: negligible load (was
>>>> undetermined)
>>>> > 63823:Jul 17 14:16:55.080 FILE-2 pacemaker-controld [15962]
>>>> (throttle_update) debug: Node FILE-2 has negligible load and supports at
>>>> most 20 jobs; new job limit 20
>>>> > 63824:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (handle_request) debug: Raising I_JOIN_RESULT: join-1
>>>> > 63825:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x00800000 (new_actions)
>>>> for controller set by s_crmd_fsa:198
>>>> > 63826:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (s_crmd_fsa) debug: Processing I_JOIN_RESULT: [ state=S_PENDING
>>>> cause=C_HA_MESSAGE origin=route_message ]
>>>> > 63827:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00800000 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63828:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (do_cl_join_finalize_respond) debug: Confirming join-1: sending local
>>>> operation history to FILE-6
>>>> > 63829:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000200
>>>> (new_actions) for controller set by s_crmd_fsa:198
>>>> > 63830:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (s_crmd_fsa) debug: Processing I_NOT_DC: [ state=S_PENDING
>>>> cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
>>>> > 63831:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000000
>>>> (an_action) for controller cleared by do_fsa_action:108
>>>> > 63832:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (do_log) info: Input I_NOT_DC received in state S_PENDING from
>>>> do_cl_join_finalize_respond
>>>> > 63833:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (do_state_transition) notice: State transition S_PENDING -> S_NOT_DC |
>>>> input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond
>>>> > 63834:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x00000020
>>>> (A_INTEGRATE_TIMER_STOP) for controller set by do_state_transition:559
>>>> > 63835:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__set_flags_as) debug: FSA action flags 0x00000080
>>>> (A_FINALIZE_TIMER_STOP) for controller set by do_state_transition:565
>>>> > 63836:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000200 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63837:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000020 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63838:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962]
>>>> (pcmk__clear_flags_as) debug: FSA action flags 0x00000080 (an_action)
>>>> for controller cleared by do_fsa_action:108
>>>> > 63863:Jul 17 14:17:25.073 FILE-2 pacemaker-controld [15962]
>>>> (throttle_cib_load) debug: cib load: 0.000667 (2 ticks in 30s)
>>>> > 63864:Jul 17 14:17:25.073 FILE-2 pacemaker-controld [15962]
>>>> (throttle_mode) debug: Current load is 0.650000 across 10 core(s)
>>>> > 63865:Jul 17 14:17:55.073 FILE-2 pacemaker-controld [15962]
>>>> (throttle_cib_load) debug: cib load: 0.000333 (1 ticks in 30s)
>>>> > 63866:Jul 17 14:17:55.073 FILE-2 pacemaker-controld [15962]
>>>> (throttle_mode) debug: Current load is 0.850000 across 10 core(s)
>>>> > 63868:Jul 17 14:18:20.085 FILE-2 pacemaker-fenced [15958]
>>>> (process_remote_stonith_exec) debug: Finalizing action 'reboot'
>>>> targeting FILE-2 on behalf of pacemaker-controld.19415 at FILE-6: OK |
>>>> rc=0 id=4e523b34
>>>> > 63869:Jul 17 14:18:20.085 FILE-2 pacemaker-fenced [15958]
>>>> (remote_op_done) notice: Operation 'reboot' targeting FILE-2 by FILE-4
>>>> for pacemaker-controld.19415 at FILE-6: OK | id=4e523b34
>>>> > 63872:Jul 17 14:18:20.089 FILE-2 pacemaker-controld [15962]
>>>> (exec_alert_list) info: Sending fencing alert via pf-ha-alert to (null)
>>>> > 63875:Jul 17 14:18:20.089 FILE-2 pacemaker-controld [15962]
>>>> (tengine_stonith_notify) crit: We were allegedly just fenced by FILE-4
>>>> for FILE-6!
>>>> > 63876:Jul 17 14:18:20.089 FILE-2 pacemaker-controld [15962]
>>>> (crm_xml_cleanup) info: Cleaning up memory from libxml2
>>>> > 63877:Jul 17 14:18:20.089 FILE-2 pacemaker-controld [15962]
>>>> (crm_exit) info: Exiting pacemaker-controld | with status 100
>>>> > 63900:Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]
>>>> (pcmk_child_exit) warning: Shutting cluster down because
>>>> pacemaker-controld[15962] had fatal failure
>>>> > 63902:Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]
>>>> (pcmk_shutdown_worker) debug: pacemaker-controld confirmed stopped
>>>> > 63956:Jul 17 14:18:20.101 FILE-2 pacemaker-fenced [15958]
>>>> (process_remote_stonith_exec) debug: Finalizing action 'reboot'
>>>> targeting FILE-1 on behalf of pacemaker-controld.19415 at FILE-6: OK |
>>>> rc=0 id=446afc42
>>>> > 63957:Jul 17 14:18:20.101 FILE-2 pacemaker-fenced [15958]
>>>> (remote_op_done) notice: Operation 'reboot' targeting FILE-1 by FILE-5
>>>> for pacemaker-controld.19415 at FILE-6: OK | id=446afc42>
>>>> > Thanks
>>>> > Priyanka
>>>>
>>>> Hi, node FILE-6 requested that node FILE-2 be fenced by node FILE-4.
>>>> FILE-2's controller daemon received notification that it was being
>>>> fenced, and it shut down. You'd want to check the logs on FILE-6 to
>>>> determine why FILE-2 was fenced.
>>>>
>>>> >
>>>> > On Thu, Jul 20, 2023 at 12:07 AM Ken Gaillot <kgaillot at redhat.com>
>>>> wrote:
>>>> >>
>>>> >> On Wed, 2023-07-19 at 23:49 +0530, Priyanka Balotra wrote:
>>>> >> > Hi All,
>>>> >> > I am using SLES 15 SP4. One of the nodes of the cluster is brought
>>>> >> > down and boot up after sometime. Pacemaker service came up first
>>>> but
>>>> >> > later it faced a fatal shutdown. Due to that crm service is down.
>>>> >> >
>>>> >> > The logs from /var/log/pacemaker.pacemaker.log are as follows:
>>>> >> >
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]
>>>> >> > (pcmk_child_exit) warning: Shutting cluster down because
>>>> >> > pacemaker-controld[15962] had fatal failure
>>>> >>
>>>> >> The interesting messages will be before this. The ones with
>>>> "pacemaker-
>>>> >> controld" will be the most relevant, at least initially.
>>>> >>
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]
>>>> >> > (pcmk_shutdown_worker) notice: Shutting down Pacemaker
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]
>>>> >> > (pcmk_shutdown_worker) debug: pacemaker-controld confirmed
>>>> stopped
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemakerd [15956] (stop_child)
>>>> >> > notice: Stopping pacemaker-schedulerd | sent signal 15 to process
>>>> >> > 15961
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemaker-schedulerd[15961]
>>>> >> > (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15
>>>> >> > (invoking handler)
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemaker-schedulerd[15961]
>>>> >> > (qb_ipcs_us_withdraw) info: withdrawing server sockets
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemaker-schedulerd[15961]
>>>> >> > (qb_ipcs_unref) debug: qb_ipcs_unref() - destroying
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemaker-schedulerd[15961]
>>>> >> > (crm_xml_cleanup) info: Cleaning up memory from libxml2
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemaker-schedulerd[15961] (crm_exit)
>>>> >> > info: Exiting pacemaker-schedulerd | with status 0
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemaker-based [15957]
>>>> >> > (qb_ipcs_event_sendv) debug: new_event_notification
>>>> (/dev/shm/qb-
>>>> >> > 15957-15962-12-RDPw6O/qb): Broken pipe (32)
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemaker-based [15957]
>>>> >> > (cib_notify_send_one) warning: Could not notify client crmd:
>>>> >> > Broken pipe | id=e29d175e-7e91-4b6a-bffb-fabfdd7a33bf
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemaker-based [15957]
>>>> >> > (cib_process_request) info: Completed cib_delete operation for
>>>> >> > section //node_state[@uname='FILE-2']/*: OK (rc=0, origin=FILE-
>>>> >> > 6/crmd/74, version=0.24.75)
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemaker-fenced [15958]
>>>> >> > (xml_patch_version_check) debug: Can apply patch 0.24.75 to
>>>> >> > 0.24.74
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]
>>>> >> > (pcmk_child_exit) info: pacemaker-schedulerd[15961] exited
>>>> >> > with status 0 (OK)
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemaker-based [15957]
>>>> >> > (cib_process_request) info: Completed cib_modify operation for
>>>> >> > section status: OK (rc=0, origin=FILE-6/crmd/75, version=0.24.75)
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]
>>>> >> > (pcmk_shutdown_worker) debug: pacemaker-schedulerd confirmed
>>>> >> > stopped
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemakerd [15956] (stop_child)
>>>> >> > notice: Stopping pacemaker-attrd | sent signal 15 to process
>>>> 15960
>>>> >> > Jul 17 14:18:20.093 FILE-2 pacemaker-attrd [15960]
>>>> >> > (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15
>>>> >> > (invoking handler)
>>>> >> >
>>>> >> > Could you please help me understand the issue here.
>>>> >> >
>>>> >> > Regards
>>>> >> > Priyanka
>>>> >> > _______________________________________________
>>>> >> > Manage your subscription:
>>>> >> > https://lists.clusterlabs.org/mailman/listinfo/users
>>>> >> >
>>>> >> > ClusterLabs home: https://www.clusterlabs.org/
>>>> >> --
>>>> >> Ken Gaillot <kgaillot at redhat.com>
>>>> >>
>>>> >> _______________________________________________
>>>> >> Manage your subscription:
>>>> >> https://lists.clusterlabs.org/mailman/listinfo/users
>>>> >>
>>>> >> ClusterLabs home: https://www.clusterlabs.org/
>>>> >
>>>> > _______________________________________________
>>>> > Manage your subscription:
>>>> > https://lists.clusterlabs.org/mailman/listinfo/users
>>>> >
>>>> > ClusterLabs home: https://www.clusterlabs.org/
>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>>
>>>> Reid Wahl (He/Him)
>>>> Senior Software Engineer, Red Hat
>>>> RHEL High Availability - Pacemaker
>>>>
>>>> _______________________________________________
>>>> Manage your subscription:
>>>> https://lists.clusterlabs.org/mailman/listinfo/users
>>>>
>>>> ClusterLabs home: https://www.clusterlabs.org/
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20230724/1d887542/attachment-0001.htm>
More information about the Users
mailing list