<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan 9, 2017 at 6:55 PM, Andrew Beekhof <span dir="ltr"><<a href="mailto:abeekhof@redhat.com" target="_blank">abeekhof@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Fri, Dec 16, 2016 at 8:52 AM, Chris Walker<br>
<<a href="mailto:christopher.walker@gmail.com">christopher.walker@gmail.com</a>> wrote:<br>
> Thanks for your response Ken. I'm puzzled ... in my case node remain<br>
> UNCLEAN (offline) until dc-deadtime expires, even when both nodes are up and<br>
> corosync is quorate.<br>
<br>
</span>I'm guessing you're starting both nodes at the same time?<br></blockquote><div><br></div><div>The nodes power on at the same time, but hardware discovery can vary by minutes.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The behaviour you're seeing is arguably a hangover from the multicast<br>
days (in which case corosync wouldn't have had a node list).<br></blockquote><div><br></div><div>That makes sense. </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
But since that's not the common case anymore, we could probably<br>
shortcut the timeout if we know the complete node list and see that<br>
they are all online.<br></blockquote><div><br></div><div>That would be ideal. It's easy enough to work around this in systemd, but it seems like the HA stack should be the authority on node status.</div><div><br></div><div>Thanks!</div><div>Chris</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5"><br>
><br>
> I see the following from crmd when I have dc-deadtime=2min<br>
><br>
> Dec 15 21:34:33 max04 crmd[13791]: notice: Quorum acquired<br>
> Dec 15 21:34:33 max04 crmd[13791]: notice: pcmk_quorum_notification: Node<br>
> max04[2886730248] - state is now member (was (null))<br>
> Dec 15 21:34:33 max04 crmd[13791]: notice: pcmk_quorum_notification: Node<br>
> (null)[2886730249] - state is now member (was (null))<br>
> Dec 15 21:34:33 max04 crmd[13791]: notice: Notifications disabled<br>
> Dec 15 21:34:33 max04 crmd[13791]: notice: The local CRM is operational<br>
> Dec 15 21:34:33 max04 crmd[13791]: notice: State transition S_STARTING -><br>
> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_started ]<br>
> ...<br>
> Dec 15 21:36:33 max05 crmd[10365]: warning: FSA: Input I_DC_TIMEOUT from<br>
> crm_timer_popped() received in state S_PENDING<br>
> Dec 15 21:36:33 max05 crmd[10365]: notice: State transition S_ELECTION -><br>
> S_INTEGRATION [ input=I_ELECTION_DC cause=C_TIMER_POPPED<br>
> origin=election_timeout_popped ]<br>
> Dec 15 21:36:33 max05 crmd[10365]: warning: FSA: Input I_ELECTION_DC from<br>
> do_election_check() received in state S_INTEGRATION<br>
> Dec 15 21:36:33 max05 crmd[10365]: notice: Notifications disabled<br>
> Dec 15 21:36:33 max04 crmd[13791]: notice: State transition S_PENDING -><br>
> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE<br>
> origin=do_cl_join_finalize_<wbr>respond ]<br>
><br>
> only after this do the nodes transition to Online. This is using the<br>
> vanilla RHEL7.2 cluster stack and the following options:<br>
><br>
> property cib-bootstrap-options: \<br>
> no-quorum-policy=ignore \<br>
> default-action-timeout=120s \<br>
> pe-warn-series-max=1500 \<br>
> pe-input-series-max=1500 \<br>
> pe-error-series-max=1500 \<br>
> stonith-action=poweroff \<br>
> stonith-timeout=900 \<br>
> dc-deadtime=2min \<br>
> maintenance-mode=false \<br>
> have-watchdog=false \<br>
> dc-version=1.1.13-10.el7-<wbr>44eb2dd \<br>
> cluster-infrastructure=<wbr>corosync<br>
><br>
> Thanks again,<br>
> Chris<br>
><br>
> On Thu, Dec 15, 2016 at 3:26 PM, Ken Gaillot <<a href="mailto:kgaillot@redhat.com">kgaillot@redhat.com</a>> wrote:<br>
>><br>
>> On 12/15/2016 02:00 PM, Chris Walker wrote:<br>
>> > Hello,<br>
>> ><br>
>> > I have a quick question about dc-deadtime. I believe that Digimer and<br>
>> > others on this list might have already addressed this, but I want to<br>
>> > make sure I'm not missing something.<br>
>> ><br>
>> > If my understanding is correct, dc-deadtime sets the amount of time that<br>
>> > must elapse before a cluster is formed (DC is elected, etc), regardless<br>
>> > of which nodes have joined the cluster. In other words, even if all<br>
>> > nodes that are explicitly enumerated in the nodelist section have<br>
>> > started Pacemaker, they will still wait dc-deadtime before forming a<br>
>> > cluster.<br>
>> ><br>
>> > In my case, I have a two-node cluster on which I'd like to allow a<br>
>> > pretty long time (~5 minutes) for both nodes to join before giving up on<br>
>> > them. However, if they both join quickly, I'd like to proceed to form a<br>
>> > cluster immediately; I don't want to wait for the full five minutes to<br>
>> > elapse before forming a cluster. Further, if a node doesn't respond<br>
>> > within five minutes, I want to fence it and start resources on the node<br>
>> > that is up.<br>
>><br>
>> Pacemaker+corosync behaves as you describe by default.<br>
>><br>
>> dc-deadtime is how long to wait for an election to finish, but if the<br>
>> election finishes sooner than that (i.e. a DC is elected), it stops<br>
>> waiting. It doesn't even wait for all nodes, just a quorum.<br>
>><br>
>> Also, with startup-fencing=true (the default), any unseen nodes will be<br>
>> fenced, and the remaining nodes will proceed to host resources. Of<br>
>> course, it needs quorum for this, too.<br>
>><br>
>> With two nodes, quorum is handled specially, but that's a different topic.<br>
>><br>
>> > With Pacemaker/Heartbeat, the initdead parameter did exactly what I<br>
>> > want, but I don't see any way to do this with Pacemaker/Corosync. From<br>
>> > reading other posts, it looks like people use an external agent to start<br>
>> > HA daemons once nodes are up ... is this a correct understanding?<br>
>> ><br>
>> > Thanks very much,<br>
>> > Chris<br>
>><br>
>> ______________________________<wbr>_________________<br>
>> Users mailing list: <a href="mailto:Users@clusterlabs.org">Users@clusterlabs.org</a><br>
>> <a href="http://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.clusterlabs.org/<wbr>mailman/listinfo/users</a><br>
>><br>
>> Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
>> Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">http://www.clusterlabs.org/<wbr>doc/Cluster_from_Scratch.pdf</a><br>
>> Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> Users mailing list: <a href="mailto:Users@clusterlabs.org">Users@clusterlabs.org</a><br>
> <a href="http://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.clusterlabs.org/<wbr>mailman/listinfo/users</a><br>
><br>
> Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
> Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">http://www.clusterlabs.org/<wbr>doc/Cluster_from_Scratch.pdf</a><br>
> Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
><br>
<br>
______________________________<wbr>_________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org">Users@clusterlabs.org</a><br>
<a href="http://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.clusterlabs.org/<wbr>mailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">http://www.clusterlabs.org/<wbr>doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
</div></div></blockquote></div><br></div></div>