[ClusterLabs] ClusterIP won't return to recovered node

[Ken Gaillot]

On 06/16/2017 01:18 PM, Dan Ragle wrote:
> 
> 
> On 6/12/2017 10:30 AM, Ken Gaillot wrote:
>> On 06/12/2017 09:23 AM, Klaus Wenninger wrote:
>>> On 06/12/2017 04:02 PM, Ken Gaillot wrote:
>>>> On 06/10/2017 10:53 AM, Dan Ragle wrote:
>>>>> So I guess my bottom line question is: How does one tell Pacemaker
>>>>> that
>>>>> the individual legs of globally unique clones should *always* be
>>>>> spread
>>>>> across the available nodes whenever possible, regardless of the number
>>>>> of processes on any one of the nodes? For kicks I did try:
>>>>>
>>>>> pcs constraint location ClusterIP:0 prefers node1-pcs=INFINITY
>>>>>
>>>>> but it responded with an error about an invalid character (:).
>>>> There isn't a way currently. It will try to do that when initially
>>>> placing them, but once they've moved together, there's no simple way to
>>>> tell them to move. I suppose a workaround might be to create a dummy
>>>> resource that you constrain to that node so it looks like the other
>>>> node
>>>> is less busy.
>>>
>>> Another ugly dummy resource idea - maybe less fragile -
>>> and not tried out:
>>> One could have 2 dummy resources that would rather like
>>> to live on different nodes - no issue with primitives - and
>>> do depend collocated on ClusterIP.
>>> Wouldn't that pull them apart once possible?
>>
>> Sounds like a good idea
> 
> Hmmmm... still no luck with this.
> 
> Based on your suggestion, I thought this would work (leaving out all the
> status displays this time):
> 
> # pcs resource create Test1 systemd:test1
> # pcs resource create Test2 systemd:test2
> # pcs constraint location Test1 prefers node1-pcs=INFINITY
> # pcs constraint location Test2 prefers node1-pcs=INFINITY
> # pcs resource create Test3 systemd:test3
> # pcs resource create Test4 systemd:test4
> # pcs constraint location Test3 prefers node1-pcs=INFINITY
> # pcs constraint location Test4 prefers node2-pcs=INFINITY
> # pcs resource create ClusterIP ocf:heartbeat:IPaddr2 ip=162.220.75.138
> nic=bond0 cidr_netmask=24
> # pcs resource meta ClusterIP resource-stickiness=0
> # pcs resource clone ClusterIP clone-max=2 clone-node-max=2
> globally-unique=true
> # pcs constraint colocation add ClusterIP-clone with Test3 INFINITY
> # pcs constraint colocation add ClusterIP-clone with Test4 INFINITY
> 
> But that simply refuses to run ClusterIP at all ("Resource ClusterIP:0/1
> cannot run anywhere"). And if I change the last two colocation
> constraints to a numeric then it runs, but with the same problem I had
> before (both ClusterIP instances on one node).
> 
> I also tried it reversing the colocation definition (add Test3 with
> ClusterIP-clone) and trying differing combinations of scores between the
> location and colocation constraints, still with no luck.
> 
> Thanks,
> 
> Dan

Ah of course, the colocation with both means they all have to run on the
same node, which is impossible.

FYI you can create dummy resources with ocf:pacemaker:Dummy so you don't
have to write your own agents.

OK, this is getting even hackier, but I'm thinking you can use
utilization for this:

http://clusterlabs.org/doc/en-US/Pacemaker/1.1-pcs/html-single/Pacemaker_Explained/index.html#idm139683960632560

* Create two dummy resources, each with a -INFINITY location preference
for one of the nodes, so each is allowed to run on only one node.

* Set the priority meta-attribute to a positive number on all your real
resources, and leave the dummies at 0 (so if the cluster can't run all
of them, it will stop the dummies first).

* Set placement-strategy=utilization.

* Define a utilization attribute, with values for each node and resource
like this:
** Set a utilization of 1 on all resources except the dummies and the
clone, so that their total utilization is N.
** Set a utilization of 100 on the dummies and the clone.
** Set a utilization capacity of 200 + N on each node.

(I'm assuming you never expect to have more than 99 other resources. If
that's not the case, just raise the 100 usage accordingly.)

With those values, if only one node is up, that node can host all the
real resources (including both clone instances), with the dummies
stopped. If both nodes are up, the only way the cluster can run all
resources (including the clone instances and dummies) is to spread the
clone instances out.

Again, it's hacky, and I haven't tested it, but I think it would work.