[ClusterLabs] cloned ethmonitor - upon failure of all nodes

[Ken Gaillot]

On Thu, 2019-08-15 at 10:59 +0100, solarmon wrote:
> Hi,
> 
> I have a two node cluster setup where each node is multi-homed over
> two separate external interfaces - net4 and net5 - that can have
> traffic load balanced between them.
> 
> I have created multiple virtual ip resources (grouped together) that
> should only be active on only one of the two nodes.
> 
> I have created ethmonitor resources for net4 and net5 and have
> created a constraint against the virtual ip resources group.
> 
> When one of the net4/net5 interfaces is taken on the active node
> (where the virtual IPs are), the virtual ip resource group switches
> to the other node. This is working as expected.
> 
> However, when either of the net4/net5 interfaces are down on BOTH
> nodes - for example, if net4 is down on BOTH nodes - the cluster
> seems to get itself in to a flapping state where there virtual IP
> resources keeps becoming available then unavailable. Or the virtual
> IP resources group isn't running on any node.
> 
> Since net4 and net5 interfaces can have traffic load-balanced across
> them, it is acceptable for the virtual IP resources to be running any
> of the node, even if the same interface (for example, net4) is down
> on both nodes, since the other interface (for example, net5) is still
> available on both nodes.
> 
> What is the recommended way to configure the ethmonitor and
> constraint resources for this type of multi-homed setup?

It's probably the constraint. When monitoring a single interface, the
location constraint should have rule giving a score of -INFINITY when
the special node attribute's value is 0.

However in your case, your goal is more complicated, so the rule has to
be as well. I'd set a -INFINITY score when *both* attributes are 0
(e.g. ethmonitor-net4 eq 0 and ethmonitor-net5 eq 0). That will keep
the IPs on a node where at least one interface is working.

If you additionally want to prefer a node with both interfaces working,
I'd add 2 more rules giving a slightly negative preference to a node
where a single attribute is 0 (one rule for each attribute).
-- 
Ken Gaillot <kgaillot at redhat.com>