[Pacemaker] Favor one node during stonith?

[Andrew Beekhof]

On 14 Aug 2014, at 1:37 am, Andrey Borzenkov <arvidjaar at gmail.com> wrote:

> В Wed, 13 Aug 2014 08:56:58 -0400
> Digimer <lists at alteeve.ca> пишет:
> 
>> On 13/08/14 08:37 AM, Andrey Borzenkov wrote:
>>> Hi,
>>> 
>>> Sorry for may be basic question, but it is my first Linux HA project.
>>> 
>>> I (will) have two node cluster in active/passive configuration -
>>> single application on one node and second as standby; application si
>>> implemented as master/slave clone. Is it possible to prioritize node,
>>> that have active application? So that in case of split brain passive
>>> node gets killed?
>>> 
>>> Usually this is done using staggered delay for fencing requests. I
>>> think that it may be possible to implement in pacemaker using rules,
>>> but I'm a bit uneasy about how to express it. Rule should select a
>>> node where master is currently active, not fixed node.
>>> 
>>> Thank you in advance!
>>> 
>>> -andrei
>> 
>> Hi Andrei,
>> 
>>   "Basic questions" is how you avoid mistakes, so please never 
>> apologize for asking them. I sure ask my own basic questions... :P
>> 
>>   First up, a little semantics; "split-brain" is what happens when 
>> fencing fails.

Not really.

Split-brain is whenever you have parts of the cluster that cant talk to other parts.
Usually it comes up when the numbers on each side of the split are the same.

Ie. 2 nodes that can talk to each other but not to 2 other node (that can also talk to each other).

You also get it any time nodes in a 2 node cluster can't talk.

Fencing is a method of _resolving_ a split-brain.

>> Your asking what happens when the connection between the 
>> nodes break when both nodes are otherwise happy (sometimes called a 
>> "partitioning of the cluster", though I don't think there is an official 
>> term).
>> 
>>   You are right in guessing that it is "delay" to set this. You add the 
>> attribute 'delay="15"' to the fence method. You put the delay attribute 
>> on the node you want to win. Here is an example;
>> 
>> http://clusterlabs.org/wiki/STONITH_Levels#Configuring_The_Fence_Methods
>> 
>>   In that example, the node called "pcmk-1" has the 'delay' set, so it 
>> will get a 15 second head start in fencing "pcmk-2". It works that way 
>> because what it does is tell the cluster "If you want to fence 'pcmk-1', 
>> pause for 15 seconds before doing so". So in a 2-node cluster 
>> partitioning, both would initiate a fence against the other immediately, 
>> but pcmk-2 would pause before fencing pcmk-1, where pcmk-1 would *not* 
>> pause before fencing pcmk-2.
>> 
> 
> 
> It statically assigns priorities to cluster nodes. I need to
> dynamically assign higher priority (lower delay) to a node that is
> currently running application to ensure that application survives. It
> was relatively easy in other cluster products I worked with, but I
> cannot figure out how to do it in pacemaker so far.

Its not something we have traditionally supported as it only really makes sense in a strict active/passive environment.
As soon as you have more than one node hosting resources, or hosting more than one resource, determining the priority quickly gets complicated.

Better to avoid split-brain in the first place.
Add a third node?  (Even if you don't allow it to run services).
That way when a node disappears, its almost certainly because its dead/failed.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://lists.clusterlabs.org/pipermail/pacemaker/attachments/20140814/28fc2fc5/attachment-0003.sig>