<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/3.16.3">
</HEAD>
<BODY>
Hi Andrew,<BR>
<BR>
<BLOCKQUOTE TYPE=CITE>
<FONT COLOR="#000000">I'd say you removed no-quorum-policy=ignore</FONT><BR>
</BLOCKQUOTE>
<BR>
Actually, the pair of no_quorum_policy and no-quorum-policy are set to "ignore", and expected-quorum-votes is set to "2":
<PRE>
<FONT SIZE="2"> <crm_config></FONT>
<FONT SIZE="2"> <cluster_property_set id="cib-bootstrap-options"></FONT>
<FONT SIZE="2"> ...</FONT>
<FONT SIZE="2"> <nvpair id="cib-bootstrap-options-expected-quorum-votes" name="expected-quorum-votes" value="2"/></FONT>
<FONT SIZE="2"> <nvpair id="cib-bootstrap-options-no_quorum_policy" name="no_quorum_policy" value="ignore"/></FONT>
<FONT SIZE="2"> <nvpair id="nvpair-1d2c923d-7619-4b45-989a-698357f9f8cb" name="no-quorum-policy" value="ignore"/></FONT>
<FONT SIZE="2"> ...</FONT>
<FONT SIZE="2"> </cluster_property_set></FONT>
<FONT SIZE="2"> </crm_config></FONT>
</PRE>
<BR>
Removing the <FONT COLOR="#000000">no-quorum-policy=ignore</FONT> and <FONT COLOR="#000000">no</FONT>_<FONT COLOR="#000000">quorum</FONT>_<FONT COLOR="#000000">policy=ignore</FONT> (as in, deleting the variables) left the cluster unable to failover with either an ifdown iface or with a node reboot. The state displayed by the GUI did not agree with the state displayed by crm_mon (the GUI showed the ifdown or rebooted node as still controlling resources, whereas crm_mon showed the resources unavailable ... both showed the inaccessible node as offline).<BR>
<BR>
Setting the no-quorum-policy=stop had the same results, which included the resources not migrating to the working system until returning <FONT COLOR="#000000">no-quorum-policy=ignore</FONT>. One of the tests led to filesystem corruption. Very messy. (this is a test-only setup, so no real data is present)<BR>
<BR>
So, no, the change that I made was neither deleting nor setting no-quorum-policy=stop. Setting <FONT COLOR="#000000">no-quorum-policy=ignore</FONT> seems to be required for the cluster to support migrations and failovers.<BR>
<BR>
Cheers and thanks,<BR>
Bob Haxo<BR>
<BR>
<BR>
On Wed, 2009-05-20 at 11:17 +0200, Andrew Beekhof wrote:
<BLOCKQUOTE TYPE=CITE>
<PRE>
<FONT COLOR="#000000">On Wed, May 20, 2009 at 1:31 AM, Bob Haxo <<A HREF="mailto:bhaxo@sgi.com">bhaxo@sgi.com</A>> wrote:</FONT>
<FONT COLOR="#000000">> Greetings,</FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">> I liked the idea of not starting the cluster at boot, and found that the</FONT>
<FONT COLOR="#000000">> fenced node would reboot and then openais start brought the node onboard</FONT>
<FONT COLOR="#000000">> without triggering a reboot of the already running node.</FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">> Then magic happened. I chkconfig'd openais to start with boot, re-ran the</FONT>
<FONT COLOR="#000000">> "ifdown eth0" command that had been triggering STONITH and then the STONITH</FONT>
<FONT COLOR="#000000">> deathmarch, and, well, everything worked. I've done this test many 10s of</FONT>
<FONT COLOR="#000000">> times without a STONITH deathmarch.</FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">> Unfortunately, I haven't a clue as to what was changed that cleared the</FONT>
<FONT COLOR="#000000">> issue.</FONT>
<FONT COLOR="#000000">At a guess, I'd say you removed no-quorum-policy=ignore</FONT>
<FONT COLOR="#000000">OpenAIS based clusters don't pretend they have quorum when only 1 of</FONT>
<FONT COLOR="#000000">the 2 nodes is available (and you cant start shooting until you have</FONT>
<FONT COLOR="#000000">quorum or the above option is set).</FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">> Thanks for all the suggestions.</FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">> Cheers,</FONT>
<FONT COLOR="#000000">> Bob Haxo</FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">> On Tue, 2009-05-19 at 14:03 +0200, Andrew Beekhof wrote:</FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">> On Mon, May 18, 2009 at 8:12 PM, Bob Haxo <<A HREF="mailto:bhaxo@sgi.com">bhaxo@sgi.com</A>> wrote:</FONT>
<FONT COLOR="#000000">>></FONT>
<FONT COLOR="#000000">>> Any suggestions as to what needs changing so that the stonith deathmarch</FONT>
<FONT COLOR="#000000">>> can</FONT>
<FONT COLOR="#000000">>> be avoided?</FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">> If you only have two nodes, the only two ways have already discussed:</FONT>
<FONT COLOR="#000000">> use poweroff, or don't start the cluster at boot.</FONT>
<FONT COLOR="#000000">> If you don't want to do either of those, the only way to terminate the</FONT>
<FONT COLOR="#000000">> stonith loop is to fix the network failure.</FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">> If you had 3 or more nodes, the returning node wouldn't have quorum</FONT>
<FONT COLOR="#000000">> and therefore wouldn't be allowed to shoot anyone.</FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">> _______________________________________________</FONT>
<FONT COLOR="#000000">> Pacemaker mailing list</FONT>
<FONT COLOR="#000000">> <A HREF="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</A></FONT>
<FONT COLOR="#000000">> <A HREF="http://oss.clusterlabs.org/mailman/listinfo/pacemaker">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</A></FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">> _______________________________________________</FONT>
<FONT COLOR="#000000">> Pacemaker mailing list</FONT>
<FONT COLOR="#000000">> <A HREF="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</A></FONT>
<FONT COLOR="#000000">> <A HREF="http://oss.clusterlabs.org/mailman/listinfo/pacemaker">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</A></FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">></FONT>
<FONT COLOR="#000000">_______________________________________________</FONT>
<FONT COLOR="#000000">Pacemaker mailing list</FONT>
<FONT COLOR="#000000"><A HREF="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</A></FONT>
<FONT COLOR="#000000"><A HREF="http://oss.clusterlabs.org/mailman/listinfo/pacemaker">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</A></FONT>
</PRE>
</BLOCKQUOTE>
</BODY>
</HTML>