[ClusterLabs] -INFINITY location constraint not honored?

Andrei Borzenkov arvidjaar at gmail.com
Fri Oct 18 14:32:31 EDT 2019 

18.10.2019 12:43, Raffaele Pantaleoni пишет:
> 
> Il 18/10/2019 10:21, Andrei Borzenkov ha scritto:
>> According to it, you have symmetric cluster (and apparently made typo
>> trying to change it)
>>
>>          <nvpair id="cib-bootstrap-options-symmetric-cluster"
>> name="symmetric-cluster" value="true"/>
>>          <nvpair id="cib-bootstrap-options-symmectric-cluster"
>> name="symmectric-cluster" value="true"/>
>>
> That's correct. I tried both approaches, opt in explicitly allowing
> nodes to be bound to resources. Then I tried the opt out strategy
> denying resource to be bound to some nodes. Both approaches failed
> though.

According to CIB you provided all nodes except one -  SRVDRSW01 - are in
standby. IOW there is really no choice where to run any resource.

So pacemaker behavior looks at least logical - it attempts to make sure
resources are started. After all, this is its primary goal.

Whether this is intentional I do not know. As already mentioned in this
discussion, the whole ordering and (co-)location can certainly benefit
from better documentation.

> My fault? It could be, obviously. But is simply follwed the
> directions found in Clusters from scratch and Configuration Explained.
> The main issue is: pcs constraint shows an apparently correct
> constrainted configuration, crm_simulate doesn't agree on such
> constraints on the other hand. :D
>> On Fri, Oct 18, 2019 at 10:29 AM Raffaele Pantaleoni
>> <r.pantaleoni at seko.com> wrote:
>> >
>> > Il 17/10/2019 18:08, Ken Gaillot ha scritto:
>> >> This does sound odd, possibly a bug. Can you provide the output of
>> "pcs
>> >> cluster cib" when one of the unexpected results is happening? (Strip
>> >> out any passwords or other sensitive information, and you can
>> e-mail it
>> >> to me privately if you don't want it on the list.)
>> >>
>> > There's no problem at all in sharing the informations. I'm working on a
>> > sandbox to test Pacemaker and then use it on a production set to
>> achieve
>> > high availability and fault tolerance, so I'm working on dummy machines
>> > inside our internal network.
>> > (meanwhile I added three nodes more, but the behaviour has not changed)
>> >
>> > Thanks
>> >> On Thu, 2019-10-17 at 16:55 +0200, Raffaele Pantaleoni wrote:
>> >>> Il 17/10/2019 16:47, Raffaele Pantaleoni ha scritto:
>> >>>> Il 17/10/2019 14:51, Jan Pokorný ha scritto:
>> >>>>> On 17/10/19 08:22 +0200, Raffaele Pantaleoni wrote:
>> >>>>>> I'm rather new to Pacemaker, I'm performing early tests on a
>> >>>>>> set of
>> >>>>>> three virtual machines.
>> >>>>>>
>> >>>>>> I am configuring the cluster in the following way:
>> >>>>>>
>> >>>>>> 3 nodes configured
>> >>>>>> 4 resources configured
>> >>>>>>
>> >>>>>> Online: [ SRVDRSW01 SRVDRSW02 SRVDRSW03 ]
>> >>>>>>
>> >>>>>>     ClusterIP      (ocf::heartbeat:IPaddr2):       Started
>> >>>>>> SRVDRSW01
>> >>>>>>     CouchIP        (ocf::heartbeat:IPaddr2):       Started
>> >>>>>> SRVDRSW03
>> >>>>>>     FrontEnd       (ocf::heartbeat:nginx): Started SRVDRSW01
>> >>>>>>     ITATESTSERVER-DIP      (ocf::nodejs:pm2):      Started
>> >>>>>> SRVDRSW01
>> >>>>>>
>> >>>>>> with the following constraints:
>> >>>>>>
>> >>>>>> Location Constraints:
>> >>>>>>      Resource: ClusterIP
>> >>>>>>        Enabled on: SRVRDSW01 (score:200)
>> >>>>>>        Enabled on: SRVRDSW02 (score:100)
>> >>>>>>      Resource: CouchIP
>> >>>>>>        Enabled on: SRVRDSW02 (score:10)
>> >>>>>>        Enabled on: SRVRDSW03 (score:100)
>> >>>>>>        Disabled on: SRVRDSW01 (score:-INFINITY)
>> >>>>>>      Resource: FrontEnd
>> >>>>>>        Enabled on: SRVRDSW01 (score:200)
>> >>>>>>        Enabled on: SRVRDSW02 (score:100)
>> >>>>>>      Resource: ITATESTSERVER-DIP
>> >>>>>>        Enabled on: SRVRDSW01 (score:200)
>> >>>>>>        Enabled on: SRVRDSW02 (score:100)
>> >>>>>> Ordering Constraints:
>> >>>>>>      start ClusterIP then start FrontEnd (kind:Mandatory)
>> >>>>>>      start ClusterIP then start ITATESTSERVER-DIP
>> >>>>>> (kind:Mandatory)
>> >>>>>> Colocation Constraints:
>> >>>>>>      FrontEnd with ClusterIP (score:INFINITY)
>> >>>>>>      FrontEnd with ITATESTSERVER-DIP (score:INFINITY)
>> >>>>>>
>> >>>>>> I've configured the cluster with an opt in strategy using the
>> >>>>>> following
>> >>>>>> commands:
>> >>>>>>
>> >>>>>> crm_attribute --name symmectric-cluster --update false
>> >>>>>>
>> >>>>>> pcs resource create ClusterIP ocf:heartbeat:IPaddr2
>> >>>>>> ip=172.16.10.126
>> >>>>>> cidr_netmask=16 op monitor interval=30s
>> >>>>>> pcs resource create CouchIP ocf:heartbeat:IPaddr2
>> >>>>>> ip=172.16.10.128
>> >>>>>> cidr_netmask=16 op monitor interval=30s
>> >>>>>> pcs resource create FrontEnd ocf:heartbeat:nginx
>> >>>>>> configfile=/etc/nginx/nginx.conf
>> >>>>>> pcs resource create ITATESTSERVER-DIP ocf:nodejs:pm2
>> >>>>>> user=ITATESTSERVER
>> >>>>>> --force
>> >>>>>>
>> >>>>>> pcs constraint colocation add FrontEnd with ClusterIP INFINITY
>> >>>>>> pcs constraint colocation add FrontEnd with ITATESTSERVER-DIP
>> >>>>>> INFINITY
>> >>>>>>
>> >>>>>> pcs constraint order ClusterIP then FrontEnd
>> >>>>>> pcs constraint order ClusterIP then ITATESTSERVER-DIP
>> >>>>>>
>> >>>>>> pcs constraint location ClusterIP prefers SRVRDSW01=200
>> >>>>>> pcs constraint location ClusterIP prefers SRVRDSW02=100
>> >>>>>>
>> >>>>>> pcs constraint location CouchIP prefers SRVRDSW02=10
>> >>>>>> pcs constraint location CouchIP prefers SRVRDSW03=100
>> >>>>>>
>> >>>>>> pcs constraint location FrontEnd prefers SRVRDSW01=200
>> >>>>>> pcs constraint location FrontEnd prefers SRVRDSW02=100
>> >>>>>>
>> >>>>>> pcs constraint location ITATESTSERVER-DIP prefers SRVRDSW01=200
>> >>>>>> pcs constraint location ITATESTSERVER-DIP prefers SRVRDSW02=100
>> >>>>>>
>> >>>>>> Everything seems to be ok but when I put vm 02 and 03 in
>> >>>>>> standby I'd
>> >>>>>> expect
>> >>>>>> CouchIP not be assigned to vm 01 beacuse of the constraint.
>> >>>>>>
>> >>>>>> The IPaddr2 resource gets assigned to vm 01 no matter what.
>> >>>>>>
>> >>>>>> Node SRVDRSW02: standby
>> >>>>>> Node SRVDRSW03: standby
>> >>>>>> Online: [ SRVDRSW01 ]
>> >>>>>>
>> >>>>>> Full list of resources:
>> >>>>>>
>> >>>>>>     ClusterIP      (ocf::heartbeat:IPaddr2):       Started
>> >>>>>> SRVDRSW01
>> >>>>>>     CouchIP        (ocf::heartbeat:IPaddr2):       Started
>> >>>>>> SRVDRSW01
>> >>>>>>     FrontEnd       (ocf::heartbeat:nginx): Started SRVDRSW01
>> >>>>>>     ITATESTSERVER-DIP      (ocf::nodejs:pm2):      Started
>> >>>>>> SRVDRSW01
>> >>>>>>
>> >>>>>> crm_simulate -sL returns the follwoing
>> >>>>>>
>> >>>>>> ---cut---
>> >>>>>>
>> >>>>>> native_color: CouchIP allocation score on SRVDRSW01: 0
>> >>>>>> native_color: CouchIP allocation score on SRVDRSW02: 0
>> >>>>>> native_color: CouchIP allocation score on SRVDRSW03: 0
>> >>>>>>
>> >>>>>> ---cut---
>> >>>>>> Why is that? I have explicitly assigned -INFINITY to CouchIP
>> >>>>>> resource
>> >>>>>> related to node SRVDRSW01 (as stated by pcs constraint:
>> >>>>>> Disabled on:
>> >>>>>> SRVRDSW01 (score:-INFINITY) ).
>> >>>>>> What am I missing or doing wrong?
>> >>>>>
>> >>>>> I am not that deep into these relationships, proper design
>> >>>>> documentation with guided examples is non-existent[*].
>> >>>>>
>> >>>>> But it occurs to me that the situation might be the inverse of
>> >>>>> what's
>> >>>>> been confusing for typical opt-out clusters:
>> >>>>>
>> >>>>>
>> https://lists.clusterlabs.org/pipermail/users/2017-April/005463.html
>> >>>>>
>> >>>>> Have you tried avoiding:
>> >>>>>
>> >>>>>>      Resource: CouchIP
>> >>>>>>        Disabled on: SRVRDSW01 (score:-INFINITY)
>> >>>>
>> >>>> Yes, I already tried that, but I did it again nevertheless since I
>> >>>> am a
>> >>>> newbie. I deleted the whole set of resources and commented out the
>> >>>> constraint from the creation script.
>> >>>> The cluster was running, then I put all the nodes in standby and
>> >>>> brought
>> >>>> SRVRDSW01 back. The CouchIP resource has been bound to the
>> >>>> "forbidden"
>> >>>> node.
>> >>>> crm_simulate -sL still gives a score of 0 to the three nodes when
>> >>>> it
>> >>>> should be something like -INFINITY 100 and 200 respectively.
>> >>>>
>> >>>> Just to make the whole thing more confusing: I noticed that
>> >>>> SRVRDSW03,
>> >>>> that is (implicitly) not allowed to get the ClusterIP resource is
>> >>>> marked
>> >>>>     (correctly) as -INFINITY from crm_simulate. So the opt in
>> >>>> configuration would seem to be correct, but for the CouchIP
>> >>>> resource
>> >>>> that is no special or different from the ClusterIP resource.
>> >>>>
>> >>>> I am really disoriented.
>> >>>>>
>> >>>
>> >>> Just another bit of information: I put the whole set in stand by
>> >>> then
>> >>> brought back SRVRDSW03... surprise surprise the ClusterIP resource
>> >>> has
>> >>> been bound to it even if it shouldn't.
>> >>>
>> >>> What's wrong?
>> >>>
>> >>>>> altogether, since when such explicit edge would be missing,
>> >>>>> implicit
>> >>>>> "cannot" (for opt-in cluster) would apply anyway, and perhaps
>> >>>>> even
>> >>>>> reliably, then?
>> >>>>>
>> >>>>>
>> >>>>> [*] as far as I know, except for
>> >>>>>
>> >>>>>
>> https://wiki.clusterlabs.org/w/images/a/ae/Ordering_Explained_-_White.pdf
>> >>>>>
>> >>>>>
>> https://wiki.clusterlabs.org/w/images/8/8a/Colocation_Explained_-_White.pdf
>>
>> >>>>>
>> >>>>>        probably not revised for giving a truthful model in all
>> >>>>> details
>> >>>>> for years,
>> >>>>>        and not demonstrating the effect of symmetry requested or
>> >>>>> avoided
>> >>>>> within
>> >>>>>        the cluster on those, amongst others
>> >>>>>        (shameless plug: there will be such coverage for upcoming
>> >>>>> group
>> >>>>> based
>> >>>>>        access control addition [those facilities haven't been
>> >>>>> terminated in
>> >>>>>        back-end so far] as a first foray in this area -- also the
>> >>>>> correct
>> >>>>>        understanding is rather important here)
>> >>>>>
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> Manage your subscription:
>> >>>>> https://lists.clusterlabs.org/mailman/listinfo/users
>> >>>>>
>> >>>>> ClusterLabs home: https://www.clusterlabs.org/
>> >>>>>
>> >>>>
>> >>>> Thank you for your reply.
>> >>>
>> >>>
>> >
>> > --
>> > Raffaele Pantaleoni
>> >
>> > _______________________________________________
>> > Manage your subscription:
>> > https://lists.clusterlabs.org/mailman/listinfo/users
>> >
>> > ClusterLabs home: https://www.clusterlabs.org/
>> _______________________________________________
>> Manage your subscription:
>> https://lists.clusterlabs.org/mailman/listinfo/users
>>
>> ClusterLabs home: https://www.clusterlabs.org/
>>
>

More information about the Users mailing list