[ClusterLabs] [Enhancement] When STONITH is not completed, a resource moves.

[renayama19661014 at ybb.ne.jp]

Hi All,

The following problem produced us in Pacemaker1.1.12.
While STONITH was not completed, a resource moved it.

The next movement seemed to happen in a cluster.

Step1) Start a cluster.

Step2) Node 1 breaks down.

Step3) Node 1 is reconnected before practice is completed from node 2 STONITH.

Step4) Repeated between Step2 and Step3.

Step5) STONITH from node 2 is not completed, but a resource moves to node 2.



There was not resource information of node 1 when I saw pe file when a resource moved in node 2.
(snip)
  <status>
    <node_state id="3232242311" uname="node1" in_ccm="false" crmd="offline" crm-debug-origin="do_state_transition" join="down" expected="down">
      <transient_attributes id="3232242311">
        <instance_attributes id="status-3232242311">
          <nvpair id="status-3232242311-last-failure-prm_XXX1" name="last-failure-prm_XXX1" value="1441957021"/>
          <nvpair id="status-3232242311-default_ping_set" name="default_ping_set" value="300"/>
          <nvpair id="status-3232242311-last-failure-prm_XXX2" name="last-failure-prm_XXX2" value="1441956891"/>
          <nvpair id="status-3232242311-shutdown" name="shutdown" value="0"/>
          <nvpair id="status-3232242311-probe_complete" name="probe_complete" value="true"/>
        </instance_attributes>
      </transient_attributes>
    </node_state>
    <node_state id="3232242312" in_ccm="true" crmd="online" crm-debug-origin="do_state_transition" uname="node2" join="member" expected="member">
      <transient_attributes id="3232242312">
        <instance_attributes id="status-3232242312">
          <nvpair id="status-3232242312-shutdown" name="shutdown" value="0"/>
          <nvpair id="status-3232242312-probe_complete" name="probe_complete" value="true"/>
          <nvpair id="status-3232242312-default_ping_set" name="default_ping_set" value="300"/>
        </instance_attributes>
      </transient_attributes>
      <lrm id="3232242312">
        <lrm_resources>
(snip)

While STONITH is not completed, the information of the node of cib is deleted and seems to be caused by the fact that cib does not have the resource information of the node.

The cause of the problem was that the communication of the cluster became unstable.
However, an action of this cluster is a problem.

This problem is not taking place in Pacemaker1.1.13 for the moment.
However, I think that it is the same processing as far as I see a source code.

Does the deletion of the node information not have to perform it after all new node information gathered?

 * crmd/callback.c
(snip)
void
peer_update_callback(enum crm_status_type type, crm_node_t * node, const void *data)
{
(snip)
     if (down) {
            const char *task = crm_element_value(down->xml, XML_LRM_ATTR_TASK);

            if (alive && safe_str_eq(task, CRM_OP_FENCE)) {
                crm_info("Node return implies stonith of %s (action %d) completed", node->uname,
                         down->id);

                st_fail_count_reset(node->uname);

                erase_status_tag(node->uname, XML_CIB_TAG_LRM, cib_scope_local);
                erase_status_tag(node->uname, XML_TAG_TRANSIENT_NODEATTRS, cib_scope_local);
                /* down->confirmed = TRUE; Only stonith-ng returning should imply completion */
                down->sent_update = TRUE;       /* Prevent tengine_stonith_callback() from calling send_stonith_update() */

(snip)


 * There is the log, but cannot attach it because the information of the user is included.
 * Please contact me by an email if you need it.


These contents are registered with Bugzilla.
 * http://bugs.clusterlabs.org/show_bug.cgi?id=5254


Best Regards,
Hideo Yamauchi.