[ClusterLabs] -INFINITY location constraint not honored?

Ken Gaillot kgaillot at redhat.com
Thu Oct 17 12:08:34 EDT 2019 

This does sound odd, possibly a bug. Can you provide the output of "pcs
cluster cib" when one of the unexpected results is happening? (Strip
out any passwords or other sensitive information, and you can e-mail it
to me privately if you don't want it on the list.)

On Thu, 2019-10-17 at 16:55 +0200, Raffaele Pantaleoni wrote:
> Il 17/10/2019 16:47, Raffaele Pantaleoni ha scritto:
> > Il 17/10/2019 14:51, Jan Pokorný ha scritto:
> > > On 17/10/19 08:22 +0200, Raffaele Pantaleoni wrote:
> > > > I'm rather new to Pacemaker, I'm performing early tests on a
> > > > set of
> > > > three virtual machines.
> > > > 
> > > > I am configuring the cluster in the following way:
> > > > 
> > > > 3 nodes configured
> > > > 4 resources configured
> > > > 
> > > > Online: [ SRVDRSW01 SRVDRSW02 SRVDRSW03 ]
> > > > 
> > > >   ClusterIP      (ocf::heartbeat:IPaddr2):       Started
> > > > SRVDRSW01
> > > >   CouchIP        (ocf::heartbeat:IPaddr2):       Started
> > > > SRVDRSW03
> > > >   FrontEnd       (ocf::heartbeat:nginx): Started SRVDRSW01
> > > >   ITATESTSERVER-DIP      (ocf::nodejs:pm2):      Started
> > > > SRVDRSW01
> > > > 
> > > > with the following constraints:
> > > > 
> > > > Location Constraints:
> > > >    Resource: ClusterIP
> > > >      Enabled on: SRVRDSW01 (score:200)
> > > >      Enabled on: SRVRDSW02 (score:100)
> > > >    Resource: CouchIP
> > > >      Enabled on: SRVRDSW02 (score:10)
> > > >      Enabled on: SRVRDSW03 (score:100)
> > > >      Disabled on: SRVRDSW01 (score:-INFINITY)
> > > >    Resource: FrontEnd
> > > >      Enabled on: SRVRDSW01 (score:200)
> > > >      Enabled on: SRVRDSW02 (score:100)
> > > >    Resource: ITATESTSERVER-DIP
> > > >      Enabled on: SRVRDSW01 (score:200)
> > > >      Enabled on: SRVRDSW02 (score:100)
> > > > Ordering Constraints:
> > > >    start ClusterIP then start FrontEnd (kind:Mandatory)
> > > >    start ClusterIP then start ITATESTSERVER-DIP
> > > > (kind:Mandatory)
> > > > Colocation Constraints:
> > > >    FrontEnd with ClusterIP (score:INFINITY)
> > > >    FrontEnd with ITATESTSERVER-DIP (score:INFINITY)
> > > > 
> > > > I've configured the cluster with an opt in strategy using the
> > > > following
> > > > commands:
> > > > 
> > > > crm_attribute --name symmectric-cluster --update false
> > > > 
> > > > pcs resource create ClusterIP ocf:heartbeat:IPaddr2
> > > > ip=172.16.10.126
> > > > cidr_netmask=16 op monitor interval=30s
> > > > pcs resource create CouchIP ocf:heartbeat:IPaddr2
> > > > ip=172.16.10.128
> > > > cidr_netmask=16 op monitor interval=30s
> > > > pcs resource create FrontEnd ocf:heartbeat:nginx
> > > > configfile=/etc/nginx/nginx.conf
> > > > pcs resource create ITATESTSERVER-DIP ocf:nodejs:pm2
> > > > user=ITATESTSERVER
> > > > --force
> > > > 
> > > > pcs constraint colocation add FrontEnd with ClusterIP INFINITY
> > > > pcs constraint colocation add FrontEnd with ITATESTSERVER-DIP
> > > > INFINITY
> > > > 
> > > > pcs constraint order ClusterIP then FrontEnd
> > > > pcs constraint order ClusterIP then ITATESTSERVER-DIP
> > > > 
> > > > pcs constraint location ClusterIP prefers SRVRDSW01=200
> > > > pcs constraint location ClusterIP prefers SRVRDSW02=100
> > > > 
> > > > pcs constraint location CouchIP prefers SRVRDSW02=10
> > > > pcs constraint location CouchIP prefers SRVRDSW03=100
> > > > 
> > > > pcs constraint location FrontEnd prefers SRVRDSW01=200
> > > > pcs constraint location FrontEnd prefers SRVRDSW02=100
> > > > 
> > > > pcs constraint location ITATESTSERVER-DIP prefers SRVRDSW01=200
> > > > pcs constraint location ITATESTSERVER-DIP prefers SRVRDSW02=100
> > > > 
> > > > Everything seems to be ok but when I put vm 02 and 03 in
> > > > standby I'd 
> > > > expect
> > > > CouchIP not be assigned to vm 01 beacuse of the constraint.
> > > > 
> > > > The IPaddr2 resource gets assigned to vm 01 no matter what.
> > > > 
> > > > Node SRVDRSW02: standby
> > > > Node SRVDRSW03: standby
> > > > Online: [ SRVDRSW01 ]
> > > > 
> > > > Full list of resources:
> > > > 
> > > >   ClusterIP      (ocf::heartbeat:IPaddr2):       Started
> > > > SRVDRSW01
> > > >   CouchIP        (ocf::heartbeat:IPaddr2):       Started
> > > > SRVDRSW01
> > > >   FrontEnd       (ocf::heartbeat:nginx): Started SRVDRSW01
> > > >   ITATESTSERVER-DIP      (ocf::nodejs:pm2):      Started
> > > > SRVDRSW01
> > > > 
> > > > crm_simulate -sL returns the follwoing
> > > > 
> > > > ---cut---
> > > > 
> > > > native_color: CouchIP allocation score on SRVDRSW01: 0
> > > > native_color: CouchIP allocation score on SRVDRSW02: 0
> > > > native_color: CouchIP allocation score on SRVDRSW03: 0
> > > > 
> > > > ---cut---
> > > > Why is that? I have explicitly assigned -INFINITY to CouchIP
> > > > resource
> > > > related to node SRVDRSW01 (as stated by pcs constraint:
> > > > Disabled on:
> > > > SRVRDSW01 (score:-INFINITY) ).
> > > > What am I missing or doing wrong?
> > > 
> > > I am not that deep into these relationships, proper design
> > > documentation with guided examples is non-existent[*].
> > > 
> > > But it occurs to me that the situation might be the inverse of
> > > what's
> > > been confusing for typical opt-out clusters:
> > > 
> > > https://lists.clusterlabs.org/pipermail/users/2017-April/005463.html
> > > 
> > > Have you tried avoiding:
> > > 
> > > >    Resource: CouchIP
> > > >      Disabled on: SRVRDSW01 (score:-INFINITY)
> > 
> > Yes, I already tried that, but I did it again nevertheless since I
> > am a 
> > newbie. I deleted the whole set of resources and commented out the 
> > constraint from the creation script.
> > The cluster was running, then I put all the nodes in standby and
> > brought
> > SRVRDSW01 back. The CouchIP resource has been bound to the
> > "forbidden" 
> > node.
> > crm_simulate -sL still gives a score of 0 to the three nodes when
> > it 
> > should be something like -INFINITY 100 and 200 respectively.
> > 
> > Just to make the whole thing more confusing: I noticed that
> > SRVRDSW03, 
> > that is (implicitly) not allowed to get the ClusterIP resource is
> > marked 
> >   (correctly) as -INFINITY from crm_simulate. So the opt in 
> > configuration would seem to be correct, but for the CouchIP
> > resource 
> > that is no special or different from the ClusterIP resource.
> > 
> > I am really disoriented.
> > > 
> 
> Just another bit of information: I put the whole set in stand by
> then 
> brought back SRVRDSW03... surprise surprise the ClusterIP resource
> has 
> been bound to it even if it shouldn't.
> 
> What's wrong?
> 
> > > altogether, since when such explicit edge would be missing,
> > > implicit
> > > "cannot" (for opt-in cluster) would apply anyway, and perhaps
> > > even
> > > reliably, then?
> > > 
> > > 
> > > [*] as far as I know, except for
> > >      
> > > https://wiki.clusterlabs.org/w/images/a/ae/Ordering_Explained_-_White.pdf
> > >      
> > > https://wiki.clusterlabs.org/w/images/8/8a/Colocation_Explained_-_White.pdf 
> > > 
> > >      probably not revised for giving a truthful model in all
> > > details 
> > > for years,
> > >      and not demonstrating the effect of symmetry requested or
> > > avoided 
> > > within
> > >      the cluster on those, amongst others
> > >      (shameless plug: there will be such coverage for upcoming
> > > group 
> > > based
> > >      access control addition [those facilities haven't been
> > > terminated in
> > >      back-end so far] as a first foray in this area -- also the
> > > correct
> > >      understanding is rather important here)
> > > 
> > > 
> > > _______________________________________________
> > > Manage your subscription:
> > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > 
> > > ClusterLabs home: https://www.clusterlabs.org/
> > > 
> > 
> > Thank you for your reply.
> 
> 
-- 
Ken Gaillot <kgaillot at redhat.com>

More information about the Users mailing list