<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Aptos;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Aptos",sans-serif;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:11.0pt;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-IN" link="#467886" vlink="#96607D" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi Tomas/Team,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We have upgraded Node2 in our PCS cluster after setting the <b>
feature-set</b> value configured to '3.17.4' on both nodes. However, the non-upgraded node (Node2) is unable to join the cluster(after pcs restart) and continues to encounter the same error.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The pacemaker.log shows that Pacemaker 2.1.6-1.el8 starts successfully, but all IPC connections (cib_ro, stonith-ng, lrmd, attrd, pengine, crmd) fail with "Connection refused" errors. Attached the error in pacemaker logs from non-upgraded
node. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>Query:<o:p></o:p></b></p>
<p class="MsoNormal">Is there a workaround or solution to allow a non-upgraded node to join a PCS cluster with version mismatch?<o:p></o:p></p>
<p class="MsoNormal">Are there any additional configuration steps we may have missed?<o:p></o:p></p>
<p class="MsoNormal">Could you please review the attached pacemaker logs and provide guidance?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Feature set value defined on both nodes after node1 upgraded.<o:p></o:p></p>
<p class="MsoNormal"><b>Node1:<o:p></o:p></b></p>
<p class="MsoNormal"><b>[root@node1 testadmin]# crm_attribute -t crm_config -n feature-set -v "3.17.4"<o:p></o:p></b></p>
<p class="MsoNormal">[root@node1 testadmin]# cibadmin --query --scope crm_config | grep feature-set<o:p></o:p></p>
<p class="MsoNormal"><nvpair id="cib-bootstrap-options-feature-set" name="feature-set" value="3.17.4"/><o:p></o:p></p>
<p class="MsoNormal">[root@node1 testadmin]# pcs --version<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">0.12.2<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>Node2:<o:p></o:p></b></p>
<p class="MsoNormal"><b>[root@node2 testadmin]# crm_attribute -t crm_config -n feature-set -v "3.17.4"<o:p></o:p></b></p>
<p class="MsoNormal">[root@node2 testadmin]# cibadmin --query --scope crm_config | grep feature-set<o:p></o:p></p>
<p class="MsoNormal"><nvpair id="cib-bootstrap-options-feature-set" name="feature-set" value="3.17.4"/><o:p></o:p></p>
<p class="MsoNormal">[root@node2 testadmin]# pcs --version<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">0.10.19<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">After restarting PCS on Node2 (non-upgraded node), we receive the following error:<o:p></o:p></p>
<p class="MsoNormal"><b>Non-upgraded Node2:<o:p></o:p></b></p>
<p class="MsoNormal"><b>[root@node2 testadmin]# cibadmin --query --scope crm_config | grep feature-set<o:p></o:p></b></p>
<p class="MsoNormal">Could not connect to the CIB: Transport endpoint is not connected<o:p></o:p></p>
<p class="MsoNormal">cibadmin: Init failed, could not perform requested operations: Transport endpoint is not connected<o:p></o:p></p>
<p class="MsoNormal">[root@node2 testadmin]#<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>Pacemaker.log:<o:p></o:p></b></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816040-29-9i478f/qb-event-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (main) notice: Starting Pacemaker 2.1.6-1.el8 | build=6fdc9deea29 features:agent-manpages compat-2.0 corosync-ge-2 default-concurrent-fencing generated-manpages monotonic nagios
ncurses remote systemd<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (qb_ipcs_us_publish) info: server name: pacemakerd<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (cluster_connect_cfg) debug: Corosync reports local node ID is 1<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (pcmk__ipc_is_authentic_process_active) info: Could not connect to cib_ro IPC: Connection refused<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (pcmk__ipc_is_authentic_process_active) info: Could not connect to stonith-ng IPC: Connection refused<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (pcmk__ipc_is_authentic_process_active) info: Could not connect to lrmd IPC: Connection refused<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (pcmk__ipc_is_authentic_process_active) info: Could not connect to attrd IPC: Connection refused<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (pcmk__ipc_is_authentic_process_active) info: Could not connect to pengine IPC: Connection refused<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (pcmk__ipc_is_authentic_process_active) info: Could not connect to crmd IPC: Connection refused<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (start_child) info: Using uid=189 and group=189 for process pacemaker-based<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (start_child) info: Forked child 816041 for process pacemaker-based<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (start_child) info: Forked child 816042 for process pacemaker-fenced<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (start_child) info: Forked child 816043 for process pacemaker-execd<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (start_child) info: Using uid=189 and group=189 for process pacemaker-attrd<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (start_child) info: Forked child 816044 for process pacemaker-attrd<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (start_child) info: Using uid=189 and group=189 for process pacemaker-schedulerd<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (start_child) info: Forked child 816045 for process pacemaker-schedulerd<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (start_child) info: Using uid=189 and group=189 for process pacemaker-controld<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (start_child) info: Forked child 816046 for process pacemaker-controld<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemakerd [816040] (main) notice: Pacemaker daemon successfully started and accepting connections<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-execd [816043] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-execd [816043] (main) notice: Starting Pacemaker local executor<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-execd [816043] (qb_ipcs_us_publish) info: server name: lrmd<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-execd [816043] (main) notice: Pacemaker local executor successfully started and accepting connections<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-execd [816043] (main) notice: OCF resource agent search path is /usr/lib/ocf/resource.d<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (main) notice: Starting Pacemaker fencer<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (crm_ipc_connect) debug: Could not establish stonith-ng IPC connection: Connection refused (111)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (get_cluster_type) info: Verifying cluster type: 'corosync'<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (get_cluster_type) info: Assuming an active 'corosync' cluster<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (crm_cluster_connect) notice: Connecting to corosync cluster infrastructure<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (get_local_nodeid) debug: Local nodeid is 1<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-Lcr6WP/qb-request-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-Lcr6WP/qb-response-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-Lcr6WP/qb-event-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (crm_get_peer) info: Created entry 7b4eebbc-ebac-412e-b2fe-fb393f385f5c/0x55ea07513410 for node node2/1 (1 total)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (crm_get_peer) info: Node 1 is now known as node2<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (st_peer_update_callback) debug: Broadcasting our uname because of node 1<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-rEDvuk/qb-request-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-rEDvuk/qb-response-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-rEDvuk/qb-event-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (pcmk__corosync_has_nodelist) debug: Corosync has node list<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-nrENvR/qb-request-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-nrENvR/qb-response-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-nrENvR/qb-event-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (crm_get_peer) info: Node 1 has uuid 1<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (crm_update_peer_proc) info: cluster_connect_cpg: Node node2[1] - corosync-cpg is now online<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (update_peer_state_iter) notice: Node node2 state is now member | nodeid=1 previous=unknown source=crm_update_peer_proc<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (st_peer_update_callback) debug: Broadcasting our uname because of node 1<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (pcmk__corosync_connect) info: Connection to corosync established<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-sXDqJq/qb-request-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-sXDqJq/qb-response-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-31-sXDqJq/qb-event-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (crm_ipc_connect) debug: Could not establish cib_rw IPC connection: Connection refused (111)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (pcmk__add_mainloop_ipc) debug: Connection to cib_rw failed: 111<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (cib_native_signon_raw) info: Could not connect to CIB manager for stonithd<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (cib_native_signon_raw) info: Connection to CIB manager for stonithd failed: Transport endpoint is not connected<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:27 node2 pacemaker-fenced [816042] (cib_native_signoff) debug: Disconnecting from the CIB manager<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:524301; real_size:528384; rb->word_size:132096<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:524301; real_size:528384; rb->word_size:132096<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:524301; real_size:528384; rb->word_size:132096<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (cib_native_signon_raw) info: Successfully connected to CIB manager for stonithd<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (setup_cib) info: Watching for fencing topology changes<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (qb_ipcs_us_publish) info: server name: stonith-ng<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (main) notice: Pacemaker fencer successfully started and accepting connections<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (pcmk_cpg_membership) info: Group stonith-ng event 0: node 1 pid 816042 joined via cpg_join<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (pcmk_cpg_membership) info: Group stonith-ng event 0: node2 (node 1 pid 816042) is member<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (init_cib_cache_cb) info: Updating device list from CIB<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (find_known_node) debug: Couldn't find node 1 with name node2<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (find_known_node) debug: Couldn't find node 2 with name node1<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (cib_devices_update) info: Updating devices to version 0.329.0<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (unpack_config) info: Startup probes: disabled (dangerous)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (unpack_config) debug: STONITH timeout: 60000<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (unpack_config) debug: STONITH of failed nodes is disabled<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (unpack_config) debug: Concurrent fencing is enabled<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (unpack_config) debug: Stop all active resources: false<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (unpack_config) notice: On loss of quorum: Ignore<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-fenced [816042] (unpack_config) warning: Blind faith: not fencing unseen nodes<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-execd [816043] (pcmk__new_client) debug: New IPC client f8cb6d63-a989-4fa3-bd48-1707a08a3ed5 for PID 816046 with uid 189 and gid 189<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-execd [816043] (handle_new_connection) debug: IPC credentials authenticated (/dev/shm/qb-816043-816046-6-c3JlVG/qb)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-execd [816043] (qb_ipcs_shm_connect) debug: connecting to client [816046]<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-execd [816043] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-execd [816043] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-execd [816043] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemaker-execd [816043] (process_lrmd_message) debug: Processed register operation from f8cb6d63-a989-4fa3-bd48-1707a08a3ed5: rc=0, reply=1, notify=0<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemakerd [816040] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemakerd [816040] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemakerd [816040] (qb_rb_open_2) debug: shm size:131085; real_size:135168; rb->word_size:33792<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemakerd [816040] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemakerd [816040] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816041-816040-11-F1285H/qb-request-cib_ro-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemakerd [816040] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816041-816040-11-F1285H/qb-response-cib_ro-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:28 node2 pacemakerd [816040] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816041-816040-11-F1285H/qb-event-cib_ro-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (pcmk_cpg_membership) info: Group stonith-ng event 1: node 2 pid 1680734 joined via cluster join<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (pcmk_cpg_membership) info: Group stonith-ng event 1: node2 (node 1 pid 816042) is member<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_rb_open_2) debug: shm size:1048589; real_size:1052672; rb->word_size:263168<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-37-ymWQzR/qb-request-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-37-ymWQzR/qb-response-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816042-37-ymWQzR/qb-event-cmap-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (crm_get_peer) info: Created entry 0268b46e-8790-40ac-8ad1-419bac7181bd/0x55ea0776d120 for node node1/2 (2 total)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (crm_get_peer) info: Node 2 is now known as node1<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (st_peer_update_callback) debug: Broadcasting our uname because of node 2<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (crm_get_peer) info: Node 2 has uuid 2<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (pcmk_cpg_membership) info: Group stonith-ng event 1: node1 (node 2 pid 1680734) is member<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (crm_update_peer_proc) info: pcmk_cpg_membership: Node node1[2] - corosync-cpg is now online<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (update_peer_state_iter) notice: Node node1 state is now member | nodeid=2 previous=unknown source=crm_update_peer_proc<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (st_peer_update_callback) debug: Broadcasting our uname because of node 2<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-execd [816043] (qb_ipcs_dispatch_connection_request) debug: HUP conn (/dev/shm/qb-816043-816046-6-c3JlVG/qb)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-execd [816043] (qb_ipcs_disconnect) debug: qb_ipcs_disconnect(/dev/shm/qb-816043-816046-6-c3JlVG/qb) state:2<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-execd [816043] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-816043-816046-6-c3JlVG/qb-response-lrmd-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-execd [816043] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-816043-816046-6-c3JlVG/qb-event-lrmd-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-execd [816043] (qb_rb_close_helper) debug: Free'ing ringbuffer: /dev/shm/qb-816043-816046-6-c3JlVG/qb-request-lrmd-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_child_exit) warning: Shutting cluster down because pacemaker-controld[816046] had fatal failure<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_shutdown_worker) notice: Shutting down Pacemaker<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_shutdown_worker) debug: pacemaker-controld confirmed stopped<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (stop_child) notice: Stopping pacemaker-schedulerd | sent signal 15 to process 816045<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_child_exit) info: pacemaker-schedulerd[816045] exited with status 0 (OK)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_shutdown_worker) debug: pacemaker-schedulerd confirmed stopped<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (stop_child) notice: Stopping pacemaker-attrd | sent signal 15 to process 816044<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_child_exit) info: pacemaker-attrd[816044] exited with status 0 (OK)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_shutdown_worker) debug: pacemaker-attrd confirmed stopped<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (stop_child) notice: Stopping pacemaker-execd | sent signal 15 to process 816043<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-execd [816043] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-execd [816043] (lrmd_exit) info: Terminating with 0 clients<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-execd [816043] (qb_ipcs_us_withdraw) info: withdrawing server sockets<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-execd [816043] (qb_ipcs_unref) debug: qb_ipcs_unref() - destroying<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-execd [816043] (crm_exit) info: Exiting pacemaker-execd | with status 0<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_child_exit) info: pacemaker-execd[816043] exited with status 0 (OK)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_shutdown_worker) debug: pacemaker-execd confirmed stopped<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (stop_child) notice: Stopping pacemaker-fenced | sent signal 15 to process 816042<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (stonith_shutdown) info: Terminating with 0 clients<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (cib_client_del_notify_callback) debug: Removing callback for cib_diff_notify events<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (cib_native_signoff) debug: Disconnecting from the CIB manager<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816041-816042-10-n4Sdqb/qb-request-cib_rw-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816041-816042-10-n4Sdqb/qb-response-cib_rw-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816041-816042-10-n4Sdqb/qb-event-cib_rw-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (cib_connection_destroy) info: Connection to the CIB manager closed<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_ipcs_us_withdraw) info: withdrawing server sockets<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (qb_ipcs_unref) debug: qb_ipcs_unref() - destroying<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemaker-fenced [816042] (crm_exit) info: Exiting pacemaker-fenced | with status 0<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_child_exit) info: pacemaker-fenced[816042] exited with status 0 (OK)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_shutdown_worker) debug: pacemaker-fenced confirmed stopped<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (stop_child) notice: Stopping pacemaker-based | sent signal 15 to process 816041<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_child_exit) info: pacemaker-based[816041] exited with status 0 (OK)<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_shutdown_worker) debug: pacemaker-based confirmed stopped<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_shutdown_worker) notice: Shutdown complete<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmk_shutdown_worker) notice: Shutting down and staying down after fatal error<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (pcmkd_shutdown_corosync) info: Asking Corosync to shut down<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (qb_ipcc_disconnect) debug: qb_ipcc_disconnect()<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816040-29-xKhHck/qb-request-cfg-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816040-29-xKhHck/qb-response-cfg-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (qb_rb_close_helper) debug: Closing ringbuffer: /dev/shm/qb-816023-816040-29-xKhHck/qb-event-cfg-header<o:p></o:p></p>
<p class="MsoNormal">Feb 13 07:15:29 node2 pacemakerd [816040] (crm_exit) info: Exiting pacemakerd | with status 100<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks and Regards,<o:p></o:p></p>
<p class="MsoNormal">S Sathish S<o:p></o:p></p>
</div>
</body>
</html>