<div dir="ltr">Sure, <div>Here are the logs: </div><div><br></div><div><br>63138:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (post_cache_update) debug: Updated cache after membership event 44.<br>63139:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x200000000 (A_ELECTION_CHECK) for controller set by post_cache_update:81<br>63140:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00000002 (an_action) for controller cleared by do_fsa_action:108<br>63141:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (do_started) info: Delaying start, Config not read (0000000000000040)<br>63142:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (register_fsa_input_adv) debug: Stalling the FSA pending further input: source=do_started cause=C_FSA_INTERNAL data=(nil) queue=0<br>63143:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x00000002 (with_actions) for controller set by register_fsa_input_adv:88<br>63144:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (s_crmd_fsa) debug: Exiting the FSA: queue=0, fsa_actions=0x200000002, stalled=true<br>63145:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (config_query_callback) debug: Call 3 : Parsing CIB options<br>63146:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (config_query_callback) debug: Shutdown escalation occurs if DC has not responded to request in 1200000ms<br>63147:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (config_query_callback) debug: Re-run scheduler after 900000ms of inactivity<br>63148:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pe_unpack_alerts) debug: Alert pf-ha-alert: path=/usr/lib/ocf/resource.d/pacemaker/pf_ha_alert.sh timeout=30000ms tstamp-format='%H:%M:%S.%06N' 0 vars<br>63149:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00000002 (an_action) for controller cleared by do_fsa_action:108<br>63150:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (do_started) debug: Init server comms<br>63151:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (qb_ipcs_us_publish) info: server name: crmd<br>63152:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (do_started) notice: Pacemaker controller successfully started and accepting connections<br>63153:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x200000000 (an_action) for controller cleared by do_fsa_action:108<br>63154:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (do_election_check) debug: Ignoring election check because we are not in an election<br>63155:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x1000000000100100 (new_actions) for controller set by s_crmd_fsa:198<br>63156:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (s_crmd_fsa) debug: Processing I_PENDING: [ state=S_STARTING cause=C_FSA_INTERNAL origin=do_started ]<br>63157:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000000 (an_action) for controller cleared by do_fsa_action:108<br>63158:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (do_log) info: Input I_PENDING received in state S_STARTING from do_started<br>63159:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (do_state_transition) notice: State transition S_STARTING -> S_PENDING | input=I_PENDING cause=C_FSA_INTERNAL origin=do_started<br>63160:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x00000020 (A_INTEGRATE_TIMER_STOP) for controller set by do_state_transition:559<br>63161:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x00000080 (A_FINALIZE_TIMER_STOP) for controller set by do_state_transition:565<br>63162:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00000020 (an_action) for controller cleared by do_fsa_action:108<br>63163:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00000080 (an_action) for controller cleared by do_fsa_action:108<br>63164:Jul 17 14:16:25.132 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00100000 (an_action) for controller cleared by do_fsa_action:108<br>63165:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962] (do_cl_join_query) debug: Querying for a DC<br>63166:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00000100 (an_action) for controller cleared by do_fsa_action:108<br>63167:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962] (controld_start_timer) debug: Started Election Trigger (inject I_DC_TIMEOUT if pops after 20000ms, source=18)<br>63168:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962] (stonith_api_signon) debug: Attempting fencer connection by pacemaker-controld with mainloop<br>63175:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792<br>63176:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792<br>63177:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792<br>63178:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958] (stonith_command) debug: Processing register 8 from client pacemaker-controld.15962 with call options 0x00000000<br>63179:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958] (stonith_command) debug: Processed register from client pacemaker-controld.15962: OK (rc=0)<br><span style="background-color:rgb(255,255,0)">63180:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962] (stonith_api_signon) debug: Connection to fencer by pacemaker-controld succeeded (registration token: 5552b1b4-f725-46ac-b239-e404cadd8d94)</span><br>63181:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958] (stonith_command) debug: Processing st_notify 9 from client pacemaker-controld.15962 with call options 0x00000000<br>63182:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958] (handle_request) debug: Enabling st_notify_disconnect callbacks for client pacemaker-controld.15962<br>63183:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958] (stonith_command) debug: Processed st_notify from client pacemaker-controld.15962: OK (rc=0)<br>63184:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958] (stonith_command) debug: Processing st_notify 10 from client pacemaker-controld.15962 with call options 0x00000000<br>63185:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958] (handle_request) debug: Enabling st_notify_fence callbacks for client pacemaker-controld.15962<br>63186:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958] (stonith_command) debug: Processed st_notify from client pacemaker-controld.15962: OK (rc=0)<br>63187:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958] (stonith_command) debug: Processing st_notify 11 from client pacemaker-controld.15962 with call options 0x00000000<br>63188:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958] (handle_request) debug: Enabling st_notify_history_synced callbacks for client pacemaker-controld.15962<br>63189:Jul 17 14:16:26.132 FILE-2 pacemaker-fenced [15958] (stonith_command) debug: Processed st_notify from client pacemaker-controld.15962: OK (rc=0)<br>63190:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962] (te_trigger_stonith_history_sync) info: Fence history will be synchronized cluster-wide within 30 seconds<br>63191:Jul 17 14:16:26.132 FILE-2 pacemaker-controld [15962] (te_connect_stonith) notice: Fencer successfully connected<br><span style="background-color:rgb(255,255,0)">63192:Jul 17 14:16:32.664 FILE-2 pacemaker-controld [15962] (quorum_notification_cb) info: Quorum retained | membership=48 members=5</span><br>63193:Jul 17 14:16:32.664 FILE-2 pacemaker-controld [15962] (quorum_notification_cb) debug: Member[0] 2<br>63194:Jul 17 14:16:32.664 FILE-2 pacemaker-controld [15962] (quorum_notification_cb) debug: Member[1] 4<br>63195:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63196:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63197:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63198:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63199:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-e4qK7U/qb-request-cmap-header<br>63200:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-e4qK7U/qb-response-cmap-header<br>63201:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-e4qK7U/qb-event-cmap-header<br>63202:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 4<br>63203:Jul 17 14:16:32.668 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 4<br>63204:Jul 17 14:16:32.672 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63205:Jul 17 14:16:32.672 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63206:Jul 17 14:16:32.672 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63209:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63210:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-YYxILU/qb-request-cmap-header<br>63211:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-YYxILU/qb-response-cmap-header<br>63212:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-YYxILU/qb-event-cmap-header<br>63213:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 4<br>63214:Jul 17 14:16:32.676 FILE-2 pacemaker-controld [15962] (quorum_notification_cb) info: Obtaining name for new node 4<br>63218:Jul 17 14:16:32.684 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63222:Jul 17 14:16:32.684 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63225:Jul 17 14:16:32.684 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63240:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63241:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-Cy8QVV/qb-request-cmap-header<br>63242:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-Cy8QVV/qb-response-cmap-header<br>63243:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-Cy8QVV/qb-event-cmap-header<br>63244:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 4<br>63245:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 4<br>63246:Jul 17 14:16:32.688 FILE-2 pacemaker-controld [15962] (quorum_notification_cb) debug: Member[2] 3<br>63259:Jul 17 14:16:32.700 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63265:Jul 17 14:16:32.700 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63267:Jul 17 14:16:32.700 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63298:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63299:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-34-0DHKhX/qb-request-cmap-header<br>63300:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-34-0DHKhX/qb-response-cmap-header<br>63301:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-34-0DHKhX/qb-event-cmap-header<br>63302:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 3<br>63303:Jul 17 14:16:32.712 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 3<br>63307:Jul 17 14:16:32.720 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63313:Jul 17 14:16:32.720 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63320:Jul 17 14:16:32.720 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63351:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63352:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-34-V0bQlV/qb-request-cmap-header<br>63353:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-34-V0bQlV/qb-response-cmap-header<br>63355:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-34-V0bQlV/qb-event-cmap-header<br>63356:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 3<br>63357:Jul 17 14:16:32.728 FILE-2 pacemaker-controld [15962] (quorum_notification_cb) info: Obtaining name for new node 3<br>63365:Jul 17 14:16:32.736 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63372:Jul 17 14:16:32.736 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63374:Jul 17 14:16:32.736 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63415:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63416:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-34-EAFzTX/qb-request-cmap-header<br>63417:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-34-EAFzTX/qb-response-cmap-header<br>63418:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-34-EAFzTX/qb-event-cmap-header<br>63419:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 3<br>63420:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 3<br>63421:Jul 17 14:16:32.748 FILE-2 pacemaker-controld [15962] (quorum_notification_cb) debug: Member[3] 6<br>63425:Jul 17 14:16:32.752 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63426:Jul 17 14:16:32.752 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63427:Jul 17 14:16:32.752 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63479:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63480:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-33-q3mFYU/qb-request-cmap-header<br>63481:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-33-q3mFYU/qb-response-cmap-header<br>63482:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-33-q3mFYU/qb-event-cmap-header<br>63483:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 6<br>63484:Jul 17 14:16:32.756 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 6<br>63485:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63486:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63487:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63490:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63491:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-EcEbfV/qb-request-cmap-header<br>63492:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-EcEbfV/qb-response-cmap-header<br>63493:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-EcEbfV/qb-event-cmap-header<br>63494:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 6<br>63495:Jul 17 14:16:32.760 FILE-2 pacemaker-controld [15962] (quorum_notification_cb) info: Obtaining name for new node 6<br>63499:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63502:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63505:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63508:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63509:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-fLk4xW/qb-request-cmap-header<br>63510:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-fLk4xW/qb-response-cmap-header<br>63511:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-fLk4xW/qb-event-cmap-header<br>63512:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 6<br>63513:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 6<br>63514:Jul 17 14:16:32.764 FILE-2 pacemaker-controld [15962] (quorum_notification_cb) debug: Member[4] 5<br>63517:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63518:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63521:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63528:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63529:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-ushXmW/qb-request-cmap-header<br>63530:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-ushXmW/qb-response-cmap-header<br>63531:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-ushXmW/qb-event-cmap-header<br>63532:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 5<br>63533:Jul 17 14:16:32.768 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 5<br>63534:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63535:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63536:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63537:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63538:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-x3qVkW/qb-request-cmap-header<br>63539:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-x3qVkW/qb-response-cmap-header<br>63540:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-x3qVkW/qb-event-cmap-header<br>63541:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 5<br>63542:Jul 17 14:16:32.772 FILE-2 pacemaker-controld [15962] (quorum_notification_cb) info: Obtaining name for new node 5<br>63543:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63544:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63545:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63546:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63547:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-gUNSFU/qb-request-cmap-header<br>63548:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-gUNSFU/qb-response-cmap-header<br>63549:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-gUNSFU/qb-event-cmap-header<br>63550:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 5<br>63551:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 5<br>63552:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (update_peer_state_iter) notice: Node (null) state is now lost | nodeid=1 previous=member source=pcmk__reap_unseen_nodes<br>63553:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (post_cache_update) debug: Updated cache after membership event 48.<br>63554:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x200000000 (A_ELECTION_CHECK) for controller set by post_cache_update:81<br>63555:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x200000000 (an_action) for controller cleared by do_fsa_action:108<br>63556:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (do_election_check) debug: Ignoring election check because we are not in an election<br>63557:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (pcmk_cpg_membership) info: Group crmd event 0: node 2 pid 15962 joined via cpg_join<br>63558:Jul 17 14:16:32.776 FILE-2 pacemaker-controld [15962] (pcmk_cpg_membership) info: Group crmd event 0: FILE-2 (node 2 pid 15962) is member<br>63559:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63560:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63561:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63564:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63565:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-5PH1gV/qb-request-cmap-header<br>63566:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-5PH1gV/qb-response-cmap-header<br>63567:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-5PH1gV/qb-event-cmap-header<br>63568:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 3<br>63569:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 3<br>63570:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (pcmk_cpg_membership) info: Group crmd event 0: peer node (node 3 pid 19250) is member<br>63571:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (crm_update_peer_proc) info: pcmk_cpg_membership: Node (null)[3] - corosync-cpg is now online<br>63572:Jul 17 14:16:32.780 FILE-2 pacemaker-controld [15962] (peer_update_callback) debug: Sending hello to node 3 so that it learns our node name<br>63573:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63574:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63575:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63576:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63577:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-QATDEV/qb-request-cmap-header<br>63578:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-QATDEV/qb-response-cmap-header<br>63579:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-QATDEV/qb-event-cmap-header<br>63580:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 4<br>63581:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 4<br>63582:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (pcmk_cpg_membership) info: Group crmd event 0: peer node (node 4 pid 19122) is member<br>63583:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (crm_update_peer_proc) info: pcmk_cpg_membership: Node (null)[4] - corosync-cpg is now online<br>63584:Jul 17 14:16:32.784 FILE-2 pacemaker-controld [15962] (peer_update_callback) debug: Sending hello to node 4 so that it learns our node name<br>63585:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63586:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63587:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63588:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63589:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-TVzR1T/qb-request-cmap-header<br>63590:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-TVzR1T/qb-response-cmap-header<br>63591:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-TVzR1T/qb-event-cmap-header<br>63592:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 5<br>63593:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 5<br>63594:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (pcmk_cpg_membership) info: Group crmd event 0: peer node (node 5 pid 19273) is member<br>63595:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (crm_update_peer_proc) info: pcmk_cpg_membership: Node (null)[5] - corosync-cpg is now online<br>63596:Jul 17 14:16:32.788 FILE-2 pacemaker-controld [15962] (peer_update_callback) debug: Sending hello to node 5 so that it learns our node name<br>63597:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63598:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63599:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<br>63600:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<br>63601:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-8LRaoV/qb-request-cmap-header<br>63602:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-8LRaoV/qb-response-cmap-header<br>63603:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-13142-15962-31-8LRaoV/qb-event-cmap-header<br>63604:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (pcmk__corosync_name) info: Unable to get node name for nodeid 6<br>63605:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (get_node_name) notice: Could not obtain a node name for corosync node with id 6<br>63606:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (pcmk_cpg_membership) info: Group crmd event 0: peer node (node 6 pid 19415) is member<br>63607:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (crm_update_peer_proc) info: pcmk_cpg_membership: Node (null)[6] - corosync-cpg is now online<br>63608:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (peer_update_callback) debug: Sending hello to node 6 so that it learns our node name<br>63609:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (get_xpath_object) debug: No match for //st_notify_history_synced in /notify<br>63610:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (stonith_api_del_notification) debug: Removing callback for st_notify_history_synced events<br>63611:Jul 17 14:16:32.792 FILE-2 pacemaker-fenced [15958] (stonith_command) debug: Processing st_notify 12 from client pacemaker-controld.15962 with call options 0x00000000<br>63612:Jul 17 14:16:32.792 FILE-2 pacemaker-fenced [15958] (handle_request) debug: Disabling st_notify_history_synced callbacks for client pacemaker-controld.15962<br>63613:Jul 17 14:16:32.792 FILE-2 pacemaker-fenced [15958] (stonith_command) debug: Processed st_notify from client pacemaker-controld.15962: OK (rc=0)<br>63614:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (tengine_stonith_history_synced) debug: Fence-history synced - cancel all timers<br>63615:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (crm_get_peer) info: Node 4 is now known as FILE-4<br>63616:Jul 17 14:16:32.792 FILE-2 pacemaker-controld [15962] (update_peer_uname) warning: Node names with capitals are discouraged, consider changing 'FILE-4'<br>63617:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962] (peer_update_callback) info: Cluster node FILE-4 is now member<br>63618:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962] (crm_get_peer) info: Node 3 is now known as FILE-3<br>63619:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962] (update_peer_uname) warning: Node names with capitals are discouraged, consider changing 'FILE-3'<br>63620:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962] (peer_update_callback) info: Cluster node FILE-3 is now member<br>63621:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962] (crm_get_peer) info: Node 5 is now known as FILE-5<br>63622:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962] (update_peer_uname) warning: Node names with capitals are discouraged, consider changing 'FILE-5'<br>63623:Jul 17 14:16:32.796 FILE-2 pacemaker-controld [15962] (peer_update_callback) info: Cluster node FILE-5 is now member<br>63640:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (crm_get_peer) info: Node 6 is now known as FILE-6<br>63641:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (update_peer_uname) warning: Node names with capitals are discouraged, consider changing 'FILE-6'<br>63642:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (peer_update_callback) info: Cluster node FILE-6 is now member<br>63643:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (handle_request) debug: Raising I_JOIN_OFFER: join-1<br>63644:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x00400200 (new_actions) for controller set by s_crmd_fsa:198<br>63645:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (s_crmd_fsa) debug: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]<br>63646:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00000200 (an_action) for controller cleared by do_fsa_action:108<br>63647:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00400000 (an_action) for controller cleared by do_fsa_action:108<br>63648:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (update_dc) info: Set DC to FILE-6 (3.11.0)<br>63649:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (pcmk__update_peer_expected) info: update_dc: Node FILE-6[6] - expected state is now member (was (null))<br>63650:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x00000200 (A_DC_TIMER_STOP) for controller set by do_cl_join_offer_respond:147<br>63651:Jul 17 14:16:32.880 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00000200 (an_action) for controller cleared by do_fsa_action:108<br>63788:Jul 17 14:16:32.884 FILE-2 pacemaker-controld [15962] (do_cib_replaced) debug: Updating the CIB after a replace: DC=false<br>63811:Jul 17 14:16:32.892 FILE-2 pacemaker-controld [15962] (join_query_callback) debug: Respond to join offer join-1 from FILE-6<br>63819:Jul 17 14:16:55.080 FILE-2 pacemaker-controld [15962] (pcmk__procfs_pid_of) info: Found pacemaker-based active as process 15957<br>63820:Jul 17 14:16:55.080 FILE-2 pacemaker-controld [15962] (throttle_cib_load) debug: Init 6 + 2 ticks at 1689603415 (100 tps)<br>63821:Jul 17 14:16:55.080 FILE-2 pacemaker-controld [15962] (throttle_mode) debug: Current load is 0.980000 across 10 core(s)<br>63822:Jul 17 14:16:55.080 FILE-2 pacemaker-controld [15962] (throttle_send_command) info: New throttle mode: negligible load (was undetermined)<br>63823:Jul 17 14:16:55.080 FILE-2 pacemaker-controld [15962] (throttle_update) debug: Node FILE-2 has negligible load and supports at most 20 jobs; new job limit 20<br>63824:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (handle_request) debug: Raising I_JOIN_RESULT: join-1<br>63825:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x00800000 (new_actions) for controller set by s_crmd_fsa:198<br>63826:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (s_crmd_fsa) debug: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]<br>63827:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00800000 (an_action) for controller cleared by do_fsa_action:108<br>63828:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (do_cl_join_finalize_respond) debug: Confirming join-1: sending local operation history to FILE-6<br>63829:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x1000000000000200 (new_actions) for controller set by s_crmd_fsa:198<br>63830:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (s_crmd_fsa) debug: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]<br>63831:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x1000000000000000 (an_action) for controller cleared by do_fsa_action:108<br>63832:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (do_log) info: Input I_NOT_DC received in state S_PENDING from do_cl_join_finalize_respond<br>63833:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (do_state_transition) notice: State transition S_PENDING -> S_NOT_DC | input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond<br>63834:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x00000020 (A_INTEGRATE_TIMER_STOP) for controller set by do_state_transition:559<br>63835:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (pcmk__set_flags_as) debug: FSA action flags 0x00000080 (A_FINALIZE_TIMER_STOP) for controller set by do_state_transition:565<br>63836:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00000200 (an_action) for controller cleared by do_fsa_action:108<br>63837:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00000020 (an_action) for controller cleared by do_fsa_action:108<br>63838:Jul 17 14:16:55.092 FILE-2 pacemaker-controld [15962] (pcmk__clear_flags_as) debug: FSA action flags 0x00000080 (an_action) for controller cleared by do_fsa_action:108<br>63863:Jul 17 14:17:25.073 FILE-2 pacemaker-controld [15962] (throttle_cib_load) debug: cib load: 0.000667 (2 ticks in 30s)<br>63864:Jul 17 14:17:25.073 FILE-2 pacemaker-controld [15962] (throttle_mode) debug: Current load is 0.650000 across 10 core(s)<br>63865:Jul 17 14:17:55.073 FILE-2 pacemaker-controld [15962] (throttle_cib_load) debug: cib load: 0.000333 (1 ticks in 30s)<br>63866:Jul 17 14:17:55.073 FILE-2 pacemaker-controld [15962] (throttle_mode) debug: Current load is 0.850000 across 10 core(s)<br>63868:Jul 17 14:18:20.085 FILE-2 pacemaker-fenced [15958] (process_remote_stonith_exec) debug: Finalizing action 'reboot' targeting FILE-2 on behalf of pacemaker-controld.19415@FILE-6: OK | rc=0 id=4e523b34<br>63869:Jul 17 14:18:20.085 FILE-2 pacemaker-fenced [15958] (remote_op_done) notice: Operation 'reboot' targeting FILE-2 by FILE-4 for pacemaker-controld.19415@FILE-6: OK | id=4e523b34<br>63872:Jul 17 14:18:20.089 FILE-2 pacemaker-controld [15962] (exec_alert_list) info: Sending fencing alert via pf-ha-alert to (null)<br>63875:Jul 17 14:18:20.089 FILE-2 pacemaker-controld [15962] (tengine_stonith_notify) crit: We were allegedly just fenced by FILE-4 for FILE-6!<br>63876:Jul 17 14:18:20.089 FILE-2 pacemaker-controld [15962] (crm_xml_cleanup) info: Cleaning up memory from libxml2<br>63877:Jul 17 14:18:20.089 FILE-2 pacemaker-controld [15962] (crm_exit) info: Exiting pacemaker-controld | with status 100<br><font style="background-color:rgb(255,255,0)" color="#000000">63900:Jul 17 14:18:20.093 FILE-2 pacemakerd [15956] (pcmk_child_exit) warning: Shutting cluster down because pacemaker-controld[15962] had fatal failure<br>63902:Jul 17 14:18:20.093 FILE-2 pacemakerd [15956] (pcmk_shutdown_worker) debug: pacemaker-controld confirmed stopped</font><br>63956:Jul 17 14:18:20.101 FILE-2 pacemaker-fenced [15958] (process_remote_stonith_exec) debug: Finalizing action 'reboot' targeting FILE-1 on behalf of pacemaker-controld.19415@FILE-6: OK | rc=0 id=446afc42<br>63957:Jul 17 14:18:20.101 FILE-2 pacemaker-fenced [15958] (remote_op_done) notice: Operation 'reboot' targeting FILE-1 by FILE-5 for pacemaker-controld.19415@FILE-6: OK | id=446afc42<br></div><div><br></div><div>Thanks</div><div>Priyanka</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 20, 2023 at 12:07 AM Ken Gaillot <<a href="mailto:kgaillot@redhat.com">kgaillot@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Wed, 2023-07-19 at 23:49 +0530, Priyanka Balotra wrote:<br>
> Hi All, <br>
> I am using SLES 15 SP4. One of the nodes of the cluster is brought<br>
> down and boot up after sometime. Pacemaker service came up first but<br>
> later it faced a fatal shutdown. Due to that crm service is down. <br>
> <br>
> The logs from /var/log/pacemaker.pacemaker.log are as follows:<br>
> <br>
> Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]<br>
> (pcmk_child_exit) warning: Shutting cluster down because<br>
> pacemaker-controld[15962] had fatal failure<br>
<br>
The interesting messages will be before this. The ones with "pacemaker-<br>
controld" will be the most relevant, at least initially.<br>
<br>
> Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]<br>
> (pcmk_shutdown_worker) notice: Shutting down Pacemaker<br>
> Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]<br>
> (pcmk_shutdown_worker) debug: pacemaker-controld confirmed stopped<br>
> Jul 17 14:18:20.093 FILE-2 pacemakerd [15956] (stop_child) <br>
> notice: Stopping pacemaker-schedulerd | sent signal 15 to process<br>
> 15961<br>
> Jul 17 14:18:20.093 FILE-2 pacemaker-schedulerd[15961]<br>
> (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15<br>
> (invoking handler)<br>
> Jul 17 14:18:20.093 FILE-2 pacemaker-schedulerd[15961]<br>
> (qb_ipcs_us_withdraw) info: withdrawing server sockets<br>
> Jul 17 14:18:20.093 FILE-2 pacemaker-schedulerd[15961]<br>
> (qb_ipcs_unref) debug: qb_ipcs_unref() - destroying<br>
> Jul 17 14:18:20.093 FILE-2 pacemaker-schedulerd[15961]<br>
> (crm_xml_cleanup) info: Cleaning up memory from libxml2<br>
> Jul 17 14:18:20.093 FILE-2 pacemaker-schedulerd[15961] (crm_exit) <br>
> info: Exiting pacemaker-schedulerd | with status 0<br>
> Jul 17 14:18:20.093 FILE-2 pacemaker-based [15957]<br>
> (qb_ipcs_event_sendv) debug: new_event_notification (/dev/shm/qb-<br>
> 15957-15962-12-RDPw6O/qb): Broken pipe (32)<br>
> Jul 17 14:18:20.093 FILE-2 pacemaker-based [15957]<br>
> (cib_notify_send_one) warning: Could not notify client crmd:<br>
> Broken pipe | id=e29d175e-7e91-4b6a-bffb-fabfdd7a33bf<br>
> Jul 17 14:18:20.093 FILE-2 pacemaker-based [15957]<br>
> (cib_process_request) info: Completed cib_delete operation for<br>
> section //node_state[@uname='FILE-2']/*: OK (rc=0, origin=FILE-<br>
> 6/crmd/74, version=0.24.75)<br>
> Jul 17 14:18:20.093 FILE-2 pacemaker-fenced [15958]<br>
> (xml_patch_version_check) debug: Can apply patch 0.24.75 to<br>
> 0.24.74<br>
> Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]<br>
> (pcmk_child_exit) info: pacemaker-schedulerd[15961] exited<br>
> with status 0 (OK)<br>
> Jul 17 14:18:20.093 FILE-2 pacemaker-based [15957]<br>
> (cib_process_request) info: Completed cib_modify operation for<br>
> section status: OK (rc=0, origin=FILE-6/crmd/75, version=0.24.75)<br>
> Jul 17 14:18:20.093 FILE-2 pacemakerd [15956]<br>
> (pcmk_shutdown_worker) debug: pacemaker-schedulerd confirmed<br>
> stopped<br>
> Jul 17 14:18:20.093 FILE-2 pacemakerd [15956] (stop_child) <br>
> notice: Stopping pacemaker-attrd | sent signal 15 to process 15960<br>
> Jul 17 14:18:20.093 FILE-2 pacemaker-attrd [15960]<br>
> (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15<br>
> (invoking handler)<br>
> <br>
> Could you please help me understand the issue here.<br>
> <br>
> Regards<br>
> Priyanka<br>
> _______________________________________________<br>
> Manage your subscription:<br>
> <a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
> <br>
> ClusterLabs home: <a href="https://www.clusterlabs.org/" rel="noreferrer" target="_blank">https://www.clusterlabs.org/</a><br>
-- <br>
Ken Gaillot <<a href="mailto:kgaillot@redhat.com" target="_blank">kgaillot@redhat.com</a>><br>
<br>
_______________________________________________<br>
Manage your subscription:<br>
<a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
<br>
ClusterLabs home: <a href="https://www.clusterlabs.org/" rel="noreferrer" target="_blank">https://www.clusterlabs.org/</a><br>
</blockquote></div>