[ClusterLabs] Antw: [EXT] Re: maximum token value (knet)

[Jan Friesse]

Ulrich,

>>>> Jan Friesse <jfriesse at redhat.com> schrieb am 11.03.2021 um 18:12 in Nachricht
> <e6bf446a-186e-053c-8bdc-921c6f7e9feb at redhat.com>:
>> Strahil,
>>> Hello all,
>>> I'm building a test cluster on RHEL8.2 and I have noticed that the cluster
>> fails to assemble ( nodes stay inquorate as if the network is not working) if
>> I set the token at 30000 or more (30s+).
> 
> 
> Hi!
> 
> I know you will be bored when I say this, but anyway:
> In old HP Service Guard the node connectivity was checked with ping/pong too, and you could specify the interval and the number lost responses that declare a node unreachable. The good thing (as 

Yes, and this is exactly what 
knet_ping_interval/knet_ping_timeout/knet_pong_count is doing :) What I 
was describing are default timeouts.

opposed to the TOTEM protocol I know) was that single missed responses 
were logged, so you did not

Yup, missed responses are logged.

just have an OK/BAD status, but also an indicator how far you are away 
from BAD status.
> 
> So you have a token timeout of 30s, and we had 3 lost responses at an interval of 7 seconds (at that time the 100Mb NIC needed about 5 seconds to renegotiate after a link failure (like unplug/replug). Recent hard- and software is somewhat faster AFAIK.

Indeed. But sadly recent hard- and software are quite usually running in 
VM on super overprovisioned bare-metal so bad things are happening, 
that's why token timeouts are (by default) set to quite big numbers.

Regards,
   Honza


> 
> Regards,
> Ulrich
> 
>>
>> Knet waits for enough pong replies for other nodes before it marks them
>> as alive and starts sending/receiving packets from them. By default it
>> needs to receive 2 pongs and ping is sent 4 times in token timeout so it
>> means 15 sec until node is considered up for 30 sec token timeout.
>>
>>> What is the maximum token value with knet ?On SLES12 (I think it was
>> corosync 1) , I used to set the token/consensus with far greater values on
>> some of our clusters.
>>
>> I'm really not aware about any arbitrary limits.
>>
>>> Best Regards,Strahil Nikolov
>>>
>>
>> Regards,
>>     Honza
>>
>>>
>>>
>>> _______________________________________________
>>> Manage your subscription:
>>> https://lists.clusterlabs.org/mailman/listinfo/users
>>>
>>> ClusterLabs home: https://www.clusterlabs.org/
>>>
>>
>> _______________________________________________
>> Manage your subscription:
>> https://lists.clusterlabs.org/mailman/listinfo/users
>>
>> ClusterLabs home: https://www.clusterlabs.org/
> 
> 
> 
> 
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
> 
> ClusterLabs home: https://www.clusterlabs.org/
>