<div dir="ltr">Has no one ever faced this issue ?</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Feb 28, 2014 at 11:51 PM, K Mehta <span dir="ltr"><<a href="mailto:kiranmehta1981@gmail.com" target="_blank">kiranmehta1981@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Yes, the issue is seen only with multi state resource. Non multi state resource work fine. Looks like is_resource_started function in utils.py does not compare resource name properly. Let fs be the resource name. is_resource_started compares fs with fs:0 and fs:1 and hence match is not found and false is returned. <div>
<br></div><div><br></div><div><div>def resource_disable(argv):</div><div> if len(argv) < 1:</div><div> utils.err("You must specify a resource to disable")</div><div><br></div><div> resource = argv[0]</div>
<div> args = ["crm_resource", "-r", argv[0], "-m", "-p", "target-role", "-v", "Stopped"]</div><div> output, retval = utils.run(args)</div><div> if retval != 0:</div>
<div> utils.err(output)</div><div><br></div><div> if "--wait" in utils.pcs_options:</div><div> wait = utils.pcs_options["--wait"]</div><div> if not wait.isdigit():</div><div> utils.err("%s is not a valid number of seconds to wait" % wait)</div>
<div> sys.exit(1)</div><div> did_stop = utils.is_resource_started(resource,int(wait),True) <<< did_stop is false</div><div><br></div><div> if did_stop:</div><div> return True</div>
<div> else:</div><div> utils.err("unable to stop: '%s', please check logs for failure information" % resource)</div><div><br></div><div><br></div><div><br></div><div><div>def is_resource_started(resource,wait,stopped=False):</div>
<div> expire_time = int(time.time()) + wait</div><div> while True:</div><div> state = getClusterState()</div><div> resources = state.getElementsByTagName("resource")</div><div> for res in resources:</div>
<div> if res.getAttribute("id") == resource: <<<< never succeeds</div><div> if (res.getAttribute("role") == "Started" and not stopped) or (res.getAttribute("role") == "Stopped" and stopped):</div>
<div> return True</div><div> break</div><div> if (expire_time < int(time.time())):</div><div> break</div><div> time.sleep(1)</div><div> return False <<< False is returned</div>
</div><div><br></div><div><br></div><div><br></div></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Feb 28, 2014 at 10:49 PM, David Vossel <span dir="ltr"><<a href="mailto:dvossel@redhat.com" target="_blank">dvossel@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><br>
<br>
<br>
<br>
----- Original Message -----<br>
> From: "K Mehta" <<a href="mailto:kiranmehta1981@gmail.com" target="_blank">kiranmehta1981@gmail.com</a>><br>
> To: "The Pacemaker cluster resource manager" <<a href="mailto:pacemaker@oss.clusterlabs.org" target="_blank">pacemaker@oss.clusterlabs.org</a>><br>
> Sent: Friday, February 28, 2014 7:05:47 AM<br>
> Subject: Re: [Pacemaker] Stopping resource using pcs<br>
><br>
</div><div>> Can anyone tell me why --wait parameter always causes pcs resource disable to<br>
> return failure though resource actually stops within time ?<br>
<br>
</div>does it only show an error with multi-state resources? It is probably a bug.<br>
<span><font color="#888888"><br>
-- Vossel<br>
</font></span><div><div><br>
><br>
><br>
> On Wed, Feb 26, 2014 at 10:45 PM, K Mehta < <a href="mailto:kiranmehta1981@gmail.com" target="_blank">kiranmehta1981@gmail.com</a> > wrote:<br>
><br>
><br>
><br>
> Deleting master resource id does not work. I see the same issue.<br>
> However, uncloning helps. Delete works after disabling and uncloning.<br>
><br>
> I see anissue in using --wait option with disable. Resources moves into<br>
> stopped state but still error an error message is printed.<br>
> When --wait option is not provided, error message is not seen<br>
><br>
> [root@sys11 ~]# pcs resource<br>
> Master/Slave Set: ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> [vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8]<br>
> Masters: [ sys11 ]<br>
> Slaves: [ sys12 ]<br>
> [root@sys11 ~]# pcs resource disable ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> --wait<br>
> Error: unable to stop: 'ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8', please<br>
> check logs for failure information<br>
> [root@sys11 ~]# pcs resource<br>
> Master/Slave Set: ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> [vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8]<br>
> Stopped: [ vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8:0<br>
> vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8:1 ]<br>
> [root@sys11 ~]# pcs resource disable ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> --wait<br>
> Error: unable to stop: 'ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8', please<br>
> check logs for failure information <<<<<error message<br>
> [root@sys11 ~]# pcs resource enable ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> [root@sys11 ~]# pcs resource<br>
> Master/Slave Set: ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> [vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8]<br>
> Masters: [ sys11 ]<br>
> Slaves: [ sys12 ]<br>
> [root@sys11 ~]# pcs resource disable ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> [root@sys11 ~]# pcs resource<br>
> Master/Slave Set: ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> [vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8]<br>
> Stopped: [ vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8:0<br>
> vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8:1 ]<br>
><br>
><br>
><br>
><br>
><br>
> On Wed, Feb 26, 2014 at 8:55 PM, David Vossel < <a href="mailto:dvossel@redhat.com" target="_blank">dvossel@redhat.com</a> > wrote:<br>
><br>
><br>
><br>
> ----- Original Message -----<br>
> > From: "Frank Brendel" < <a href="mailto:frank.brendel@eurolog.com" target="_blank">frank.brendel@eurolog.com</a> ><br>
> > To: <a href="mailto:pacemaker@oss.clusterlabs.org" target="_blank">pacemaker@oss.clusterlabs.org</a><br>
> > Sent: Wednesday, February 26, 2014 8:53:19 AM<br>
> > Subject: Re: [Pacemaker] Stopping resource using pcs<br>
> ><br>
> > I guess we need some real experts here.<br>
> ><br>
> > I think it's because you're attempting to delete the resource and not the<br>
> > Master.<br>
> > Try deleting the Master instead of the resource.<br>
><br>
> Yes, delete the Master resource id, not the primitive resource within the<br>
> master. When using pcs, you should always refer to the resource's top most<br>
> parent id, not the id of the children resources within the parent. If you<br>
> make a resource a clone, start using the clone id. Same with master. If you<br>
> add a resource to a group, reference the group id from then on and not any<br>
> of the children resources within the group.<br>
><br>
> As a general practice, it is always better to stop a resource (pcs resource<br>
> disable) and only delete the resource after the stop has completed.<br>
><br>
> This is especially important for group resources where stop order matters. If<br>
> you delete a group, then we have no information on what order to stop the<br>
> resources in that group. This can cause stop failures when the orphaned<br>
> resources are cleaned up.<br>
><br>
> Recently pcs gained the ability to attempt to stop resources before deleting<br>
> them in order to avoid scenarios like i described above. Pcs will block for<br>
> a period of time waiting for the resource to stop before deleting it. Even<br>
> with this logic in place it is preferred to stop the resource manually then<br>
> delete the resource once you have verified it stopped.<br>
><br>
> -- Vossel<br>
><br>
> ><br>
> > I had a similar problem with a cloned group and solved it by un-cloning<br>
> > before deleting the group.<br>
> > Maybe un-cloning the multi-state resource could help too.<br>
> > It's easy to reproduce.<br>
> ><br>
> > # pcs resource create resPing ping host_list="10.0.0.1 10.0.0.2" op monitor<br>
> > on-fail="restart"<br>
> > # pcs resource group add groupPing resPing<br>
> > # pcs resource clone groupPing clone-max=3 clone-node-max=1<br>
> > # pcs resource<br>
> > Clone Set: groupPing-clone [groupPing]<br>
> > Started: [ node1 node2 node3 ]<br>
> > # pcs resource delete groupPing-clone<br>
> > Deleting Resource (and group) - resPing<br>
> > Error: Unable to remove resource 'resPing' (do constraints exist?)<br>
> > # pcs resource unclone groupPing<br>
> > # pcs resource delete groupPing<br>
> > Removing group: groupPing (and all resources within group)<br>
> > Stopping all resources in group: groupPing...<br>
> > Deleting Resource (and group) - resPing<br>
> ><br>
> > Log:<br>
> > Feb 26 15:43:16 node1 cibadmin[2368]: notice: crm_log_args: Invoked:<br>
> > /usr/sbin/cibadmin -o resources -D --xml-text <group id="groupPing">#012<br>
> > <primitive class="ocf" id="resPing" provider="pacemaker" type="ping">#012<br>
> > <instance_attributes id="resPing-instance_attributes">#012 <nvpair<br>
> > id="resPing-instance_attributes-host_list" name="host_list" value="10.0.0.1<br>
> > 10.0.0.2"/>#012 </instance_attributes>#012 <operations>#012 <op<br>
> > id="resPing-monitor-on-fail-restart" interval="60s" name="monitor"<br>
> > on-fail="restart"/>#012 </operations>#012 </primi<br>
> > Feb 26 15:43:16 node1 cib[1820]: error: xml_log: Expecting an element<br>
> > meta_attributes, got nothing<br>
> > Feb 26 15:43:16 node1 cib[1820]: error: xml_log: Invalid sequence in<br>
> > interleave<br>
> > Feb 26 15:43:16 node1 cib[1820]: error: xml_log: Element clone failed to<br>
> > validate content<br>
> > Feb 26 15:43:16 node1 cib[1820]: error: xml_log: Element resources has<br>
> > extra<br>
> > content: primitive<br>
> > Feb 26 15:43:16 node1 cib[1820]: error: xml_log: Invalid sequence in<br>
> > interleave<br>
> > Feb 26 15:43:16 node1 cib[1820]: error: xml_log: Element cib failed to<br>
> > validate content<br>
> > Feb 26 15:43:16 node1 cib[1820]: warning: cib_perform_op: Updated CIB does<br>
> > not validate against pacemaker-1.2 schema/dtd<br>
> > Feb 26 15:43:16 node1 cib[1820]: warning: cib_diff_notify: Update (client:<br>
> > cibadmin, call:2): 0.516.7 -> 0.517.1 (Update does not conform to the<br>
> > configured schema)<br>
> > Feb 26 15:43:16 node1 stonith-ng[1821]: warning: update_cib_cache_cb:<br>
> > [cib_diff_notify] ABORTED: Update does not conform to the configured schema<br>
> > (-203)<br>
> > Feb 26 15:43:16 node1 cib[1820]: warning: cib_process_request: Completed<br>
> > cib_delete operation for section resources: Update does not conform to the<br>
> > configured schema (rc=-203, origin=local/cibadmin/2, version=0.516.7)<br>
> ><br>
> ><br>
> > Frank<br>
> ><br>
> > Am <a href="tel:26.02.2014%2015" value="+12602201415" target="_blank">26.02.2014 15</a> :00, schrieb K Mehta:<br>
> ><br>
> ><br>
> ><br>
> > Here is the config and output of few commands<br>
> ><br>
> > [root@sys11 ~]# pcs config<br>
> > Cluster Name: kpacemaker1.1<br>
> > Corosync Nodes:<br>
> ><br>
> > Pacemaker Nodes:<br>
> > sys11 sys12<br>
> ><br>
> > Resources:<br>
> > Master: ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> > Meta Attrs: clone-max=2 globally-unique=false target-role=Started<br>
> > Resource: vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8 (class=ocf<br>
> > provider=heartbeat type=vgc-cm-agent.ocf)<br>
> > Attributes: cluster_uuid=de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> > Operations: monitor interval=30s role=Master timeout=100s<br>
> > (vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8-monitor-interval-30s)<br>
> > monitor interval=31s role=Slave timeout=100s<br>
> > (vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8-monitor-interval-31s)<br>
> ><br>
> > Stonith Devices:<br>
> > Fencing Levels:<br>
> ><br>
> > Location Constraints:<br>
> > Resource: ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> > Enabled on: sys11 (score:200)<br>
> > (id:location-ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8-sys11-200)<br>
> > Enabled on: sys12 (score:200)<br>
> > (id:location-ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8-sys12-200)<br>
> > Resource: vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> > Enabled on: sys11 (score:200)<br>
> > (id:location-vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8-sys11-200)<br>
> > Enabled on: sys12 (score:200)<br>
> > (id:location-vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8-sys12-200)<br>
> > Ordering Constraints:<br>
> > Colocation Constraints:<br>
> ><br>
> > Cluster Properties:<br>
> > cluster-infrastructure: cman<br>
> > dc-version: 1.1.8-7.el6-394e906<br>
> > no-quorum-policy: ignore<br>
> > stonith-enabled: false<br>
> > symmetric-cluster: false<br>
> ><br>
> ><br>
> ><br>
> > [root@sys11 ~]# pcs resource<br>
> > Master/Slave Set: ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> > [vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8]<br>
> > Masters: [ sys11 ]<br>
> > Slaves: [ sys12 ]<br>
> ><br>
> ><br>
> ><br>
> > [root@sys11 ~]# pcs resource disable<br>
> > vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> ><br>
> > [root@sys11 ~]# pcs resource<br>
> > Master/Slave Set: ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> > [vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8]<br>
> > Stopped: [ vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8:0<br>
> > vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8:1 ]<br>
> ><br>
> ><br>
> > [root@sys11 ~]# pcs resource delete<br>
> > vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> > Removing Constraint -<br>
> > location-ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8-sys11-200<br>
> > Removing Constraint -<br>
> > location-ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8-sys12-200<br>
> > Removing Constraint -<br>
> > location-vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8-sys11-200<br>
> > Removing Constraint -<br>
> > location-vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8-sys12-200<br>
> > Attempting to stop: vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8...Error:<br>
> > Unable<br>
> > to stop: vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8 before deleting (re-run<br>
> > with --force to force deletion)<br>
> ><br>
> ><br>
> > [root@sys11 ~]# pcs resource delete<br>
> > vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> > Attempting to stop: vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8...Error:<br>
> > Unable<br>
> > to stop: vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8 before deleting (re-run<br>
> > with --force to force deletion)<br>
> ><br>
> > [root@sys11 ~]# pcs resource<br>
> > Master/Slave Set: ms-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> > [vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8]<br>
> > Stopped: [ vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8:0<br>
> > vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8:1 ]<br>
> ><br>
> > [root@sys11 ~]# pcs resource delete<br>
> > vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> > --force<br>
> > Deleting Resource - vha-de5566b1-c2a3-4dc6-9712-c82bb43f19d8<br>
> > [root@sys11 ~]# pcs resource<br>
> > NO resources configured<br>
> > [root@sys11 ~]#<br>
> ><br>
> ><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org" target="_blank">Pacemaker@oss.clusterlabs.org</a><br>
> > <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
> ><br>
> > Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> > Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > Bugs: <a href="http://bugs.clusterlabs.org" target="_blank">http://bugs.clusterlabs.org</a><br>
> ><br>
><br>
> _______________________________________________<br>
> Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org" target="_blank">Pacemaker@oss.clusterlabs.org</a><br>
> <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
><br>
> Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> Bugs: <a href="http://bugs.clusterlabs.org" target="_blank">http://bugs.clusterlabs.org</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org" target="_blank">Pacemaker@oss.clusterlabs.org</a><br>
> <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
><br>
> Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> Bugs: <a href="http://bugs.clusterlabs.org" target="_blank">http://bugs.clusterlabs.org</a><br>
><br>
<br>
_______________________________________________<br>
Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org" target="_blank">Pacemaker@oss.clusterlabs.org</a><br>
<a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" target="_blank">http://bugs.clusterlabs.org</a><br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>