<div dir="ltr">Any pointers on this would be helpful.<div><br></div><div>Regards</div><div>Arjun<br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan 12, 2015 at 7:24 PM, Arjun Pandey <span dir="ltr"><<a href="mailto:apandepublic@gmail.com" target="_blank">apandepublic@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>I am running a 2 node cluster(active-standby mode) with this config <br></div><div><br></div><div>CentOS-6.5</div><div>pacemaker 1.1.10-14.el6_5.3<br></div><div><div>pcs 0.9.90-2.el6.centos.3 - (none) noarch</div></div><div>corosync 1.4.1-17.el6_5.1<br></div><div><br></div><div>Current cluster status:</div><div>Online: [ lakshman shatrughan ]</div><div><br></div><div> Master/Slave Set: foo-master [foo]</div><div> Masters: [ shatrughan ]</div><div> Slaves: [ lakshman ]</div><div> eth1-CP<span style="white-space:pre-wrap"> </span>(ocf::pw:IPaddr):<span style="white-space:pre-wrap"> </span>Started shatrughan </div><div> eth2-UP<span style="white-space:pre-wrap"> </span>(ocf::pw:IPaddr):<span style="white-space:pre-wrap"> </span>Started shatrughan </div><div> eth3-UPCP<span style="white-space:pre-wrap"> </span>(ocf::pw:IPaddr):<span style="white-space:pre-wrap"> </span>Started shatrughan </div><div><br></div><div><span style="font-size:12.6666669845581px">where IPaddr RA is just modified IPAddr2 RA and foo is a multi-state resource. Additionally i have a</span><br style="font-size:12.6666669845581px"><font face="arial, sans-serif" style="font-size:13px"><span style="font-size:12.6666669845581px">collocation constraint for the IP addr to be collocated with the master.</span></font><br style="font-size:12.6666669845581px"><span style="font-size:12.6666669845581px">I have set the migration-threshold as 2 for the VIP. I also have set the failure-timeout to 15s.</span><br></div><div><span style="font-size:12.6666669845581px">All virtual IP's have following properties </span></div><div><div><span style="font-size:12.5px"> Meta Attrs: failure-timeout=3s migration-threshold=2 </span></div><div><span style="font-size:12.5px"> Operations: monitor interval=1s (RAN-monitor-interval-1s)</span></div></div><div><span style="font-size:12.5px"><br></span></div><div><span style="font-size:12.5px">However after some time i notice that the colocation constraint i had applied has got removed and the resource get's started where foo standby is located. </span></div><div><span style="font-size:12.5px">From the logs </span></div><div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27112] lakshman pengine: notice: process_pe_message: <span style="white-space:pre-wrap"> </span>Calculated Transition 3: /var/lib/pacemaker/pengine/pe-input-1297.bz2</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27113] lakshman crmd: info: do_state_transition: <span style="white-space:pre-wrap"> </span>State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SFOOCESS cause=C_IPC_MESSAGE origin=handle_response ]</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27113] lakshman crmd: info: do_te_invoke: <span style="white-space:pre-wrap"> </span>Processing graph 3 (ref=pe_calc-dc-1418813132-33) derived from /var/lib/pacemaker/pengine/pe-input-1297.bz2</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27113] lakshman crmd: notice: te_rsc_command: <span style="white-space:pre-wrap"> </span>Initiating action 12: monitor foo_monitor_10000 on lakshman (local)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27113] lakshman crmd: info: do_lrm_rsc_op: <span style="white-space:pre-wrap"> </span>Performing key=12:3:8:62cbe40f-174d-458b-8e52-8581e57245a0 op=foo_monitor_10000</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27113] lakshman crmd: notice: process_lrm_event: <span style="white-space:pre-wrap"> </span>LRM operation foo_monitor_10000 (call=77, rc=8, cib-update=52, confirmed=false) master</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_process_request: <span style="white-space:pre-wrap"> </span>Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/52, version=0.652.73)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27113] lakshman crmd: info: match_graph_event: <span style="white-space:pre-wrap"> </span>Action foo_monitor_10000 (12) confirmed on lakshman (rc=0)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27113] lakshman crmd: notice: run_graph: <span style="white-space:pre-wrap"> </span>Transition 3 (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1297.bz2): Complete</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27113] lakshman crmd: info: do_log: <span style="white-space:pre-wrap"> </span>FSA: Input I_TE_SFOOCESS from notify_crmd() received in state S_TRANSITION_ENGINE</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27113] lakshman crmd: notice: do_state_transition: <span style="white-space:pre-wrap"> </span>State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SFOOCESS cause=C_FSA_INTERNAL origin=notify_crmd ]</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: <span style="white-space:pre-wrap"> </span>Connecting 0x20bb1d0 for uid=0 gid=0 pid=28098 id=4d1ef70d-3f75-4308-9c1b-cc410a6e6940</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_process_request: <span style="white-space:pre-wrap"> </span>Completed cib_query operation for section 'all': OK (rc=0, origin=local/crm_resource/2, version=0.652.73)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: <span style="white-space:pre-wrap"> </span>Destroying 0 events</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: <span style="white-space:pre-wrap"> </span>Connecting 0x20bb1d0 for uid=0 gid=0 pid=28102 id=14b4a917-4df1-46ed-82e3-0a023b92ed11</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_process_request: <span style="white-space:pre-wrap"> </span>Completed cib_query operation for section //master[@id="Mgmt_Float"]: No such device or address (rc=-6, origin=local/cibadmin/2, version=0.652.73)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: <span style="white-space:pre-wrap"> </span>Destroying 0 events</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: <span style="white-space:pre-wrap"> </span>Connecting 0x20bb1d0 for uid=0 gid=0 pid=28103 id=4ab59dd4-7f13-4ae6-84cf-ae9c2de5e246</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_process_request: <span style="white-space:pre-wrap"> </span>Completed cib_query operation for section //group[@id="Mgmt_Float"]: No such device or address (rc=-6, origin=local/cibadmin/2, version=0.652.73)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: <span style="white-space:pre-wrap"> </span>Destroying 0 events</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: <span style="white-space:pre-wrap"> </span>Connecting 0x20bb1d0 for uid=0 gid=0 pid=28104 id=7d24f5e0-2ae2-47be-a88e-1635e1268d3c</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_process_request: <span style="white-space:pre-wrap"> </span>Completed cib_query operation for section //clone[@id="Mgmt_Float"]: No such device or address (rc=-6, origin=local/cibadmin/2, version=0.652.73)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: <span style="white-space:pre-wrap"> </span>Destroying 0 events</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: <span style="white-space:pre-wrap"> </span>Connecting 0x20bb1d0 for uid=0 gid=0 pid=28105 id=c583545c-e590-4f5a-a865-e063d21f35cd</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_process_request: <span style="white-space:pre-wrap"> </span>Completed cib_query operation for section //group/primitive[@id="Mgmt_Float"]/..: No such device or address (rc=-6, origin=local/cibadmin/2, version=0.652.73)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: <span style="white-space:pre-wrap"> </span>Destroying 0 events</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: <span style="white-space:pre-wrap"> </span>Connecting 0x20bb1d0 for uid=0 gid=0 pid=28106 id=d7dc6094-4c48-4246-94b3-dc2b0a12866f</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_process_request: <span style="white-space:pre-wrap"> </span>Completed cib_query operation for section //master/primitive[@id="Mgmt_Float"]/..: No such device or address (rc=-6, origin=local/cibadmin/2, version=0.652.73)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: <span style="white-space:pre-wrap"> </span>Destroying 0 events</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: <span style="white-space:pre-wrap"> </span>Connecting 0x20bb1d0 for uid=0 gid=0 pid=28107 id=757a2e15-a1d4-4f3b-b93a-0051b5a9be15</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_process_request: <span style="white-space:pre-wrap"> </span>Completed cib_query operation for section //clone/primitive[@id="Mgmt_Float"]/..: No such device or address (rc=-6, origin=local/cibadmin/2, version=0.652.73)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: <span style="white-space:pre-wrap"> </span>Destroying 0 events</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: <span style="white-space:pre-wrap"> </span>Connecting 0x20bb1d0 for uid=0 gid=0 pid=28108 id=1c9bfb99-bd55-42dc-b1bd-df32d560af90</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_process_request: <span style="white-space:pre-wrap"> </span>Completed cib_query operation for section //resources/descendant::primitive[@id="Mgmt_Float"]: OK (rc=0, origin=local/cibadmin/2, version=0.652.73)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: <span style="white-space:pre-wrap"> </span>Destroying 0 events</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: <span style="white-space:pre-wrap"> </span>Connecting 0x20bb1d0 for uid=0 gid=0 pid=28109 id=1b7ede71-66ec-43ad-9754-b204f367ccc3</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_process_request: <span style="white-space:pre-wrap"> </span>Completed cib_query operation for section 'all': OK (rc=0, origin=local/cibadmin/2, version=0.652.73)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: <span style="white-space:pre-wrap"> </span>Destroying 0 events</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: <span style="white-space:pre-wrap"> </span>Connecting 0x20bb1d0 for uid=0 gid=0 pid=28110 id=ebd97f0f-81fc-4fd3-adae-30fb9784b563</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_process_request: <span style="white-space:pre-wrap"> </span>Completed cib_query operation for section //constraints: OK (rc=0, origin=local/cibadmin/2, version=0.652.73)</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: <span style="white-space:pre-wrap"> </span>Destroying 0 events</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: <span style="white-space:pre-wrap"> </span>Connecting 0x20bb1d0 for uid=0 gid=0 pid=28111 id=e4891335-7a9b-4189-ba3d-03008b905664</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: info: cib_replace_notify: <span style="white-space:pre-wrap"> </span>Replaced: 0.652.73 -> 0.653.1 from lakshman</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: notice: cib:diff: <span style="white-space:pre-wrap"> </span>Diff: --- 0.652.73</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: notice: cib:diff: <span style="white-space:pre-wrap"> </span>Diff: +++ 0.653.1 dfc649135fe2d861910a19c9f983cc4a</span></div><div><span style="font-size:12.5px">Dec 17 16:15:32 [27108] lakshman cib: notice: cib:diff: <span style="white-space:pre-wrap"> </span>-- <rsc_colocation id="colocation-Mgmt_Float-foo-master-INFINITY" rsc="Mgmt_Float" rsc-role="Started" score="INFINITY" with-rsc="foo-master" with-rsc-role="Master"/></span></div><div><span style="font-size:12.5px">PFA the cluster-log file</span></div><div><span style="font-size:12.5px"><br></span></div><div><span style="font-size:12.5px">I can't really make out why the constraint got removed in the first place.</span></div></div><div><span style="font-size:12.5px"><div>Thanks for help in advance.</div><div><br></div><div>Regards</div><span class="HOEnZb"><font color="#888888"><div>Arjun</div></font></span></span></div></div>
</blockquote></div><br></div></div></div>