[ClusterLabs] [Problem] When a group resource does not stop in a trouble node, the movement of the group resource is started in other nodes.

[renayama19661014 at ybb.ne.jp]

Hi All, After Pacemaker1.1.14, there may be a problem in order of the stop of the group
resource. The problem occurs by cluster constitution without STONITH. I can confirm it in the next procedure. Step 1) Copy Dummy resource and make Dummy1 resource and Dummy2 resource.
Step 2) Constitute a cluster. [root at rh72-01 ~]# crm_mon -1 -Af
Stack: corosync
Current DC: rh72-02 (version 1.1.15-e174ec8) - partition with quorum
Last updated: Wed Oct  5 16:24:21 2016          Last change: Wed Oct  5
16:24:15 2016 by root via cibadmin on rh72-01 2 nodes and 2 resources configured Online: [ rh72-01 rh72-02 ] Resource Group: grpDummy prmDummy1  (ocf::pacemaker:Dummy1):        Started rh72-01 prmDummy2  (ocf::pacemaker:Dummy2):        Started rh72-01 Node Attributes:
* Node rh72-01:
* Node rh72-02: Migration Summary:
* Node rh72-01:
* Node rh72-02: Step 3) Set pseudotrouble in stop of Dummy2.
(snip)
dummy_stop() {
return $OCF_ERR_GENERIC dummy_monitor if [ $? -eq $OCF_SUCCESS ]; then rm ${OCF_RESKEY_state} fi rm -f "${VERIFY_SERIALIZED_FILE}" return $OCF_SUCCESS
}
(snip) Step 4) Make rh72-01 node standby. Trouble occurs in Dummy2 resource, and the
resource does not move. [root at rh72-01 ~]# crm_standby -N rh72-01 -v on
[root at rh72-01 ~]# crm_mon -1 -Af
Stack: corosync
Current DC: rh72-02 (version 1.1.15-e174ec8) - partition with quorum
Last updated: Wed Oct  5 16:27:49 2016          Last change: Wed Oct  5
16:27:47 2016 by root via crm_attribute on rh72-01 2 nodes and 2 resources configured Node rh72-01: standby
Online: [ rh72-02 ] Resource Group: grpDummy prmDummy1  (ocf::pacemaker:Dummy1):        Started rh72-01 prmDummy2  (ocf::pacemaker:Dummy2):        FAILED rh72-01 (blocked) Node Attributes:
* Node rh72-01:
* Node rh72-02: Migration Summary:
* Node rh72-01: prmDummy2: migration-threshold=1 fail-count=1000000 last-failure='Wed Oct  5
16:29:29 2016'
* Node rh72-02: Failed Actions:
* prmDummy2_stop_0 on rh72-01 'unknown error' (1): call=15, status=complete,
exitreason='none', last-rc-change='Wed Oct  5 16:27:47 2016', queued=1ms, exec=34ms Step 5) Clean Dummy2 resource. [root at rh72-01 ~]# crm_resource -C -r prmDummy2 -H rh72-01 -f
Cleaning up prmDummy2 on rh72-01, removing fail-count-prmDummy2
Waiting for 1 replies from the CRMd. OK
[root at rh72-01 ~]# crm_mon -1 -Af
Stack: corosync
Current DC: rh72-02 (version 1.1.15-e174ec8) - partition with quorum
Last updated: Wed Oct  5 16:30:55 2016          Last change: Wed Oct  5
16:30:53 2016 by hacluster via crmd on rh72-01 2 nodes and 2 resources configured Node rh72-01: standby
Online: [ rh72-02 ] Resource Group: grpDummy prmDummy1  (ocf::pacemaker:Dummy1):        Started rh72-02 prmDummy2  (ocf::pacemaker:Dummy2):        FAILED rh72-01 (blocked) Node Attributes:
* Node rh72-01:
* Node rh72-02: Migration Summary:
* Node rh72-01: prmDummy2: migration-threshold=1 fail-count=1000000 last-failure='Wed Oct  5
16:32:35 2016'
* Node rh72-02: Failed Actions:
* prmDummy2_stop_0 on rh72-01 'unknown error' (1): call=23, status=complete,
exitreason='none', last-rc-change='Wed Oct  5 16:30:54 2016', queued=0ms, exec=35ms Trouble occurs again, and the Dummy2 resource does not move, but the Dummy1
resource moves to rh72-02 node.
If all the resources of the group do not stop, the resource should not move. The problem does not occur in Pacemaker1.1.13. An event of probe_complete is abolished by Pacemaker1.1.14.
It is thought that a problem is included near the next correction. * https://github.com/ClusterLabs/pacemaker/commit/c1438ae489d791cc689625332b8ced21bfd4d143#diff-8e7ae81c93497126538c2a82fe183692 * https://github.com/ClusterLabs/pacemaker/commit/8f76b782133857b40a583e947d743d45c7d05dc8#diff-8e7ae81c93497126538c2a82fe183692 

I registered this problem with Bugzilla.
* http://bugs.clusterlabs.org/show_bug.cgi?id=5301
Best Regards,
Hideo Yamauch.