[ClusterLabs] how to set a dedicated fence delay for a stonith agent ?

[Klaus Wenninger]

On 05/17/2017 01:24 PM, Lentes, Bernd wrote:
>
> ----- On May 10, 2017, at 9:15 PM, Dimitri Maziuk dmaziuk at bmrb.wisc.edu wrote:
>
>> On 05/10/2017 01:54 PM, Ken Gaillot wrote:
>>> On 05/10/2017 12:26 PM, Dimitri Maziuk wrote:
>>>> - fencing in 2-node clusters does not work reliably without fixed delay
>>> Not quite. Fixed delay allows a particular method for avoiding a death
>>> match in a two-node cluster. Pacemaker's built-in random delay
>>> capability is another method.
>> Deathmatch is one problem, killing the wrong node (2 nodes, no quorum)
>> is another. Fixed delay is digimer's attempt to alleviate the latter,
>> so... apples and fruits not entirely unlike apples.
>>
>> --
> Hi,
>
> so what should i do ? Using pcmk_delay_max does not seem to be really reliable.
> I don't like the idea of being dependent from a software thinking "which delay i should choose, depending on the ... weather conditions, any mood ..."
> I'd like to know what the software is use is doing. Am i the only one having that opinion ?
>
> How do you solve the problem of a deathmatch or killing the wrong node ?

When you just have a fence-agent available that doesn't support an
extra delay-parameter there is another solution - although not very
beautiful in my mind:

You can use fencing-levels and a dummy fence agent like the
one coming with cts (fence_dummy).
If you put it on the same level as your real fence-agent you would
have it in the list before that one, configure it to wait for a certain
time and succeed afterwards.
Or you are using multiple levels put it into a level that has
higher prio than the one your real fence-agent is in. Again make
it wait but afterwards fail so that the next level is attempted.

But I think we should aim for a solution inside pacemaker here.
I'm btw. the one that brought up a delay derived from the node
health Ken had mentioned.
I could as well imagine other enhancements to what we have
with pcmk_delay_max like a constant base delay where the
random comes on top.
It might be useful to be able to derive both - the constant base
and the random part - from attributes.
For the health-based delay I had in mind to have 3 parameters
like pcmk_delay_green, pcmk_delay_orange, pcmk_delay_red.

Maybe a good opportunity here to ask for some feedback
regarding enhancements of generic fencing delay mechanisms...

Regards,
Klaus

>
> Bernd
>  
>
> Helmholtz Zentrum Muenchen
> Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH)
> Ingolstaedter Landstr. 1
> 85764 Neuherberg
> www.helmholtz-muenchen.de
> Aufsichtsratsvorsitzende: MinDir'in Baerbel Brumme-Bothe
> Geschaeftsfuehrer: Prof. Dr. Guenther Wess, Heinrich Bassler, Dr. Alfons Enhsen
> Registergericht: Amtsgericht Muenchen HRB 6466
> USt-IdNr: DE 129521671
>
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://lists.clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org