[ClusterLabs] -INFINITY location constraint not honored?

Andrei Borzenkov arvidjaar at gmail.com
Fri Oct 18 04:21:53 EDT 2019 

According to it, you have symmetric cluster (and apparently made typo
trying to change it)

        <nvpair id="cib-bootstrap-options-symmetric-cluster"
name="symmetric-cluster" value="true"/>
        <nvpair id="cib-bootstrap-options-symmectric-cluster"
name="symmectric-cluster" value="true"/>

On Fri, Oct 18, 2019 at 10:29 AM Raffaele Pantaleoni
<r.pantaleoni at seko.com> wrote:
>
> Il 17/10/2019 18:08, Ken Gaillot ha scritto:
> > This does sound odd, possibly a bug. Can you provide the output of "pcs
> > cluster cib" when one of the unexpected results is happening? (Strip
> > out any passwords or other sensitive information, and you can e-mail it
> > to me privately if you don't want it on the list.)
> >
> There's no problem at all in sharing the informations. I'm working on a
> sandbox to test Pacemaker and then use it on a production set to achieve
> high availability and fault tolerance, so I'm working on dummy machines
> inside our internal network.
> (meanwhile I added three nodes more, but the behaviour has not changed)
>
> Thanks
> > On Thu, 2019-10-17 at 16:55 +0200, Raffaele Pantaleoni wrote:
> >> Il 17/10/2019 16:47, Raffaele Pantaleoni ha scritto:
> >>> Il 17/10/2019 14:51, Jan Pokorný ha scritto:
> >>>> On 17/10/19 08:22 +0200, Raffaele Pantaleoni wrote:
> >>>>> I'm rather new to Pacemaker, I'm performing early tests on a
> >>>>> set of
> >>>>> three virtual machines.
> >>>>>
> >>>>> I am configuring the cluster in the following way:
> >>>>>
> >>>>> 3 nodes configured
> >>>>> 4 resources configured
> >>>>>
> >>>>> Online: [ SRVDRSW01 SRVDRSW02 SRVDRSW03 ]
> >>>>>
> >>>>>    ClusterIP      (ocf::heartbeat:IPaddr2):       Started
> >>>>> SRVDRSW01
> >>>>>    CouchIP        (ocf::heartbeat:IPaddr2):       Started
> >>>>> SRVDRSW03
> >>>>>    FrontEnd       (ocf::heartbeat:nginx): Started SRVDRSW01
> >>>>>    ITATESTSERVER-DIP      (ocf::nodejs:pm2):      Started
> >>>>> SRVDRSW01
> >>>>>
> >>>>> with the following constraints:
> >>>>>
> >>>>> Location Constraints:
> >>>>>     Resource: ClusterIP
> >>>>>       Enabled on: SRVRDSW01 (score:200)
> >>>>>       Enabled on: SRVRDSW02 (score:100)
> >>>>>     Resource: CouchIP
> >>>>>       Enabled on: SRVRDSW02 (score:10)
> >>>>>       Enabled on: SRVRDSW03 (score:100)
> >>>>>       Disabled on: SRVRDSW01 (score:-INFINITY)
> >>>>>     Resource: FrontEnd
> >>>>>       Enabled on: SRVRDSW01 (score:200)
> >>>>>       Enabled on: SRVRDSW02 (score:100)
> >>>>>     Resource: ITATESTSERVER-DIP
> >>>>>       Enabled on: SRVRDSW01 (score:200)
> >>>>>       Enabled on: SRVRDSW02 (score:100)
> >>>>> Ordering Constraints:
> >>>>>     start ClusterIP then start FrontEnd (kind:Mandatory)
> >>>>>     start ClusterIP then start ITATESTSERVER-DIP
> >>>>> (kind:Mandatory)
> >>>>> Colocation Constraints:
> >>>>>     FrontEnd with ClusterIP (score:INFINITY)
> >>>>>     FrontEnd with ITATESTSERVER-DIP (score:INFINITY)
> >>>>>
> >>>>> I've configured the cluster with an opt in strategy using the
> >>>>> following
> >>>>> commands:
> >>>>>
> >>>>> crm_attribute --name symmectric-cluster --update false
> >>>>>
> >>>>> pcs resource create ClusterIP ocf:heartbeat:IPaddr2
> >>>>> ip=172.16.10.126
> >>>>> cidr_netmask=16 op monitor interval=30s
> >>>>> pcs resource create CouchIP ocf:heartbeat:IPaddr2
> >>>>> ip=172.16.10.128
> >>>>> cidr_netmask=16 op monitor interval=30s
> >>>>> pcs resource create FrontEnd ocf:heartbeat:nginx
> >>>>> configfile=/etc/nginx/nginx.conf
> >>>>> pcs resource create ITATESTSERVER-DIP ocf:nodejs:pm2
> >>>>> user=ITATESTSERVER
> >>>>> --force
> >>>>>
> >>>>> pcs constraint colocation add FrontEnd with ClusterIP INFINITY
> >>>>> pcs constraint colocation add FrontEnd with ITATESTSERVER-DIP
> >>>>> INFINITY
> >>>>>
> >>>>> pcs constraint order ClusterIP then FrontEnd
> >>>>> pcs constraint order ClusterIP then ITATESTSERVER-DIP
> >>>>>
> >>>>> pcs constraint location ClusterIP prefers SRVRDSW01=200
> >>>>> pcs constraint location ClusterIP prefers SRVRDSW02=100
> >>>>>
> >>>>> pcs constraint location CouchIP prefers SRVRDSW02=10
> >>>>> pcs constraint location CouchIP prefers SRVRDSW03=100
> >>>>>
> >>>>> pcs constraint location FrontEnd prefers SRVRDSW01=200
> >>>>> pcs constraint location FrontEnd prefers SRVRDSW02=100
> >>>>>
> >>>>> pcs constraint location ITATESTSERVER-DIP prefers SRVRDSW01=200
> >>>>> pcs constraint location ITATESTSERVER-DIP prefers SRVRDSW02=100
> >>>>>
> >>>>> Everything seems to be ok but when I put vm 02 and 03 in
> >>>>> standby I'd
> >>>>> expect
> >>>>> CouchIP not be assigned to vm 01 beacuse of the constraint.
> >>>>>
> >>>>> The IPaddr2 resource gets assigned to vm 01 no matter what.
> >>>>>
> >>>>> Node SRVDRSW02: standby
> >>>>> Node SRVDRSW03: standby
> >>>>> Online: [ SRVDRSW01 ]
> >>>>>
> >>>>> Full list of resources:
> >>>>>
> >>>>>    ClusterIP      (ocf::heartbeat:IPaddr2):       Started
> >>>>> SRVDRSW01
> >>>>>    CouchIP        (ocf::heartbeat:IPaddr2):       Started
> >>>>> SRVDRSW01
> >>>>>    FrontEnd       (ocf::heartbeat:nginx): Started SRVDRSW01
> >>>>>    ITATESTSERVER-DIP      (ocf::nodejs:pm2):      Started
> >>>>> SRVDRSW01
> >>>>>
> >>>>> crm_simulate -sL returns the follwoing
> >>>>>
> >>>>> ---cut---
> >>>>>
> >>>>> native_color: CouchIP allocation score on SRVDRSW01: 0
> >>>>> native_color: CouchIP allocation score on SRVDRSW02: 0
> >>>>> native_color: CouchIP allocation score on SRVDRSW03: 0
> >>>>>
> >>>>> ---cut---
> >>>>> Why is that? I have explicitly assigned -INFINITY to CouchIP
> >>>>> resource
> >>>>> related to node SRVDRSW01 (as stated by pcs constraint:
> >>>>> Disabled on:
> >>>>> SRVRDSW01 (score:-INFINITY) ).
> >>>>> What am I missing or doing wrong?
> >>>>
> >>>> I am not that deep into these relationships, proper design
> >>>> documentation with guided examples is non-existent[*].
> >>>>
> >>>> But it occurs to me that the situation might be the inverse of
> >>>> what's
> >>>> been confusing for typical opt-out clusters:
> >>>>
> >>>> https://lists.clusterlabs.org/pipermail/users/2017-April/005463.html
> >>>>
> >>>> Have you tried avoiding:
> >>>>
> >>>>>     Resource: CouchIP
> >>>>>       Disabled on: SRVRDSW01 (score:-INFINITY)
> >>>
> >>> Yes, I already tried that, but I did it again nevertheless since I
> >>> am a
> >>> newbie. I deleted the whole set of resources and commented out the
> >>> constraint from the creation script.
> >>> The cluster was running, then I put all the nodes in standby and
> >>> brought
> >>> SRVRDSW01 back. The CouchIP resource has been bound to the
> >>> "forbidden"
> >>> node.
> >>> crm_simulate -sL still gives a score of 0 to the three nodes when
> >>> it
> >>> should be something like -INFINITY 100 and 200 respectively.
> >>>
> >>> Just to make the whole thing more confusing: I noticed that
> >>> SRVRDSW03,
> >>> that is (implicitly) not allowed to get the ClusterIP resource is
> >>> marked
> >>>    (correctly) as -INFINITY from crm_simulate. So the opt in
> >>> configuration would seem to be correct, but for the CouchIP
> >>> resource
> >>> that is no special or different from the ClusterIP resource.
> >>>
> >>> I am really disoriented.
> >>>>
> >>
> >> Just another bit of information: I put the whole set in stand by
> >> then
> >> brought back SRVRDSW03... surprise surprise the ClusterIP resource
> >> has
> >> been bound to it even if it shouldn't.
> >>
> >> What's wrong?
> >>
> >>>> altogether, since when such explicit edge would be missing,
> >>>> implicit
> >>>> "cannot" (for opt-in cluster) would apply anyway, and perhaps
> >>>> even
> >>>> reliably, then?
> >>>>
> >>>>
> >>>> [*] as far as I know, except for
> >>>>
> >>>> https://wiki.clusterlabs.org/w/images/a/ae/Ordering_Explained_-_White.pdf
> >>>>
> >>>> https://wiki.clusterlabs.org/w/images/8/8a/Colocation_Explained_-_White.pdf
> >>>>
> >>>>       probably not revised for giving a truthful model in all
> >>>> details
> >>>> for years,
> >>>>       and not demonstrating the effect of symmetry requested or
> >>>> avoided
> >>>> within
> >>>>       the cluster on those, amongst others
> >>>>       (shameless plug: there will be such coverage for upcoming
> >>>> group
> >>>> based
> >>>>       access control addition [those facilities haven't been
> >>>> terminated in
> >>>>       back-end so far] as a first foray in this area -- also the
> >>>> correct
> >>>>       understanding is rather important here)
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> Manage your subscription:
> >>>> https://lists.clusterlabs.org/mailman/listinfo/users
> >>>>
> >>>> ClusterLabs home: https://www.clusterlabs.org/
> >>>>
> >>>
> >>> Thank you for your reply.
> >>
> >>
>
> --
> Raffaele Pantaleoni
>
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
>
> ClusterLabs home: https://www.clusterlabs.org/

More information about the Users mailing list