[ClusterLabs] Antw: Re: Antw: [EXT] Re: Sub‑clusters / super‑clusters?

[Ulrich Windl]

>>> "Frank D. Engel, Jr." <fde101 at fjrhome.net> schrieb am 05.08.2021 um 00:11 in
Nachricht <da797157-8631-f79c-829a-4a9d1792fc92 at fjrhome.net>:
> In theory if you could have an independent voting infrastructure among 
> the three clusters which serves to effectively create a second cluster 
> infrastructure interconnecting them to support resource D, you could 
> have D running on one of the clusters so long as at least two of them 
> can communicate with each other.

Hi!

That's what I thought too, BUT:
If yoiu have some common (NAS) storage where each cluster sends a "proof of life" periodically, what will happen if the network is down?
Each cluster will thing the other is dead, so no help.
Maybe it's time for an independent communication channel.
Maybe quorum-voting via SMS or packet radio? ;-)

Regards,
Ulrich
> 
> 
> In other words, give each cluster one vote, then as long as two of them 
> can communicate there are two votes which makes quorum, thus resource D 
> can run on one of those two clusters.
> 
> If all three clusters lose contact with each other, then D still cannot 
> safely run.
> 
> 
> To keep the remaining resources working when contact is lost between the 
> clusters, the vote for this would need to be independent of the vote 
> within each individual cluster, effectively meaning that each node would 
> belong to two clusters at once: its own local cluster (A/B/C) plus a 
> "global" cluster spread across the three locations.  I don't know 
> offhand if that is readily possible to support with the current software.
> 
> 
> On 8/4/21 5:01 PM, Antony Stone wrote:
>> On Wednesday 04 August 2021 at 22:06:39, Frank D. Engel, Jr. wrote:
>>
>>> There is no safe way to do what you are trying to do.
>>>
>>> If the resource is on cluster A and contact is lost between clusters A
>>> and B due to a network failure, how does cluster B know if the resource
>>> is still running on cluster A or not?
>>>
>>> It has no way of knowing if cluster A is even up and running.
>>>
>>> In that situation it cannot safely start the resource.
>> I am perfectly happy to have an additional machine at a third location in
>> order to avoid this split-brain between two clusters.
>>
>> However, what I cannot have is for the resources which should be running on
>> cluster A to get started on cluster B.
>>
>> If cluster A is down, then its resources should simply not run - as happens
>> right now with two independent clusters.
>>
>> Suppose for a moment I had three clusters at three locations: A, B and C.
>>
>> Is there a method by which I can have:
>>
>> 1. Cluster A resources running on cluster A if cluster A is functional and 
> not
>> running anywhere if cluster A is non-functional.
>>
>> 2. Cluster B resources running on cluster B if cluster B is functional and 
> not
>> running anywhere if cluster B is non-functional.
>>
>> 3. Cluster C resources running on cluster C if cluster C is functional and 
> not
>> running anywhere if cluster C is non-functional.
>>
>> 4. Resource D running _somewhere_ on clusters A, B or C, but only a single
>> instance of D at a single location at any time.
>>
>> Requirements 1, 2 and 3 are easy to achieve - don't connect the clusters.
>>
>> Requirement 4 is the one I'm stuck with how to implement.
>>
>> If the three nodes comprising cluster A can manage resources such that they
>> run on only one of the three nodes at any time, surely there must be a way 
> of
>> doing the same thing with a resource running on one of three clusters?
>>
>>
>> Antony.
>>
> 
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users 
> 
> ClusterLabs home: https://www.clusterlabs.org/