[ClusterLabs] Antw: [EXT] Re: Sub‑clusters / super‑clusters?

[Frank D. Engel, Jr.]

In theory if you could have an independent voting infrastructure among 
the three clusters which serves to effectively create a second cluster 
infrastructure interconnecting them to support resource D, you could 
have D running on one of the clusters so long as at least two of them 
can communicate with each other.


In other words, give each cluster one vote, then as long as two of them 
can communicate there are two votes which makes quorum, thus resource D 
can run on one of those two clusters.

If all three clusters lose contact with each other, then D still cannot 
safely run.


To keep the remaining resources working when contact is lost between the 
clusters, the vote for this would need to be independent of the vote 
within each individual cluster, effectively meaning that each node would 
belong to two clusters at once: its own local cluster (A/B/C) plus a 
"global" cluster spread across the three locations.  I don't know 
offhand if that is readily possible to support with the current software.


On 8/4/21 5:01 PM, Antony Stone wrote:
> On Wednesday 04 August 2021 at 22:06:39, Frank D. Engel, Jr. wrote:
>
>> There is no safe way to do what you are trying to do.
>>
>> If the resource is on cluster A and contact is lost between clusters A
>> and B due to a network failure, how does cluster B know if the resource
>> is still running on cluster A or not?
>>
>> It has no way of knowing if cluster A is even up and running.
>>
>> In that situation it cannot safely start the resource.
> I am perfectly happy to have an additional machine at a third location in
> order to avoid this split-brain between two clusters.
>
> However, what I cannot have is for the resources which should be running on
> cluster A to get started on cluster B.
>
> If cluster A is down, then its resources should simply not run - as happens
> right now with two independent clusters.
>
> Suppose for a moment I had three clusters at three locations: A, B and C.
>
> Is there a method by which I can have:
>
> 1. Cluster A resources running on cluster A if cluster A is functional and not
> running anywhere if cluster A is non-functional.
>
> 2. Cluster B resources running on cluster B if cluster B is functional and not
> running anywhere if cluster B is non-functional.
>
> 3. Cluster C resources running on cluster C if cluster C is functional and not
> running anywhere if cluster C is non-functional.
>
> 4. Resource D running _somewhere_ on clusters A, B or C, but only a single
> instance of D at a single location at any time.
>
> Requirements 1, 2 and 3 are easy to achieve - don't connect the clusters.
>
> Requirement 4 is the one I'm stuck with how to implement.
>
> If the three nodes comprising cluster A can manage resources such that they
> run on only one of the three nodes at any time, surely there must be a way of
> doing the same thing with a resource running on one of three clusters?
>
>
> Antony.
>