[ClusterLabs] Preferred location is sometimes ignored

[Ken Gaillot]

On 09/16/2016 09:45 AM, Auer, Jens wrote:
> Hi,
> 
> I have a Active/Passive configuration with a drbd mast/slave resource:
> 
> MDA1PFP-S01 14:40:27 1803 0 ~ # pcs status
> Cluster name: MDA1PFP
> Last updated: Fri Sep 16 14:41:18 2016        Last change: Fri Sep 16
> 14:39:49 2016 by root via cibadmin on MDA1PFP-PCS01
> Stack: corosync
> Current DC: MDA1PFP-PCS02 (version 1.1.13-10.el7-44eb2dd) - partition
> with quorum
> 2 nodes and 7 resources configured
> 
> Online: [ MDA1PFP-PCS01 MDA1PFP-PCS02 ]
> 
> Full list of resources:
> 
>  Master/Slave Set: drbd1_sync [drbd1]
>      Masters: [ MDA1PFP-PCS02 ]
>      Slaves: [ MDA1PFP-PCS01 ]
>  mda-ip    (ocf::heartbeat:IPaddr2):    Started MDA1PFP-PCS02
>  Clone Set: ping-clone [ping]
>      Started: [ MDA1PFP-PCS01 MDA1PFP-PCS02 ]
>  ACTIVE    (ocf::heartbeat:Dummy):    Started MDA1PFP-PCS02
>  shared_fs    (ocf::heartbeat:Filesystem):    Started MDA1PFP-PCS02
> 
> PCSD Status:
>   MDA1PFP-PCS01: Online
>   MDA1PFP-PCS02: Online
> 
> Daemon Status:
>   corosync: active/disabled
>   pacemaker: active/disabled
>   pcsd: active/enabled
> 
> MDA1PFP-S01 14:41:19 1804 0 ~ # pcs resource --full
>  Master: drbd1_sync
>   Meta Attrs: master-max=1 master-node-max=1 clone-max=2
> clone-node-max=1 notify=true
>   Resource: drbd1 (class=ocf provider=linbit type=drbd)
>    Attributes: drbd_resource=shared_fs
>    Operations: start interval=0s timeout=240 (drbd1-start-interval-0s)
>                promote interval=0s timeout=90 (drbd1-promote-interval-0s)
>                demote interval=0s timeout=90 (drbd1-demote-interval-0s)
>                stop interval=0s timeout=100 (drbd1-stop-interval-0s)
>                monitor interval=60s (drbd1-monitor-interval-60s)
>  Resource: mda-ip (class=ocf provider=heartbeat type=IPaddr2)
>   Attributes: ip=192.168.120.20 cidr_netmask=32 nic=bond0
>   Operations: start interval=0s timeout=20s (mda-ip-start-interval-0s)
>               stop interval=0s timeout=20s (mda-ip-stop-interval-0s)
>               monitor interval=1s (mda-ip-monitor-interval-1s)
>  Clone: ping-clone
>   Resource: ping (class=ocf provider=pacemaker type=ping)
>    Attributes: dampen=5s multiplier=1000 host_list=pf-pep-dev-1
> timeout=1 attempts=3
>    Operations: start interval=0s timeout=60 (ping-start-interval-0s)
>                stop interval=0s timeout=20 (ping-stop-interval-0s)
>                monitor interval=1 (ping-monitor-interval-1)
>  Resource: ACTIVE (class=ocf provider=heartbeat type=Dummy)
>   Operations: start interval=0s timeout=20 (ACTIVE-start-interval-0s)
>               stop interval=0s timeout=20 (ACTIVE-stop-interval-0s)
>               monitor interval=10 timeout=20 (ACTIVE-monitor-interval-10)
>  Resource: shared_fs (class=ocf provider=heartbeat type=Filesystem)
>   Attributes: device=/dev/drbd1 directory=/shared_fs fstype=xfs
>   Operations: start interval=0s timeout=60 (shared_fs-start-interval-0s)
>               stop interval=0s timeout=60 (shared_fs-stop-interval-0s)
>               monitor interval=20 timeout=40 (shared_fs-monitor-interval-20)
> 
> MDA1PFP-S01 14:41:35 1805 0 ~ # pcs constraint --full
> Location Constraints:
>   Resource: mda-ip
>     Enabled on: MDA1PFP-PCS01 (score:50)
> (id:location-mda-ip-MDA1PFP-PCS01-50)
>     Constraint: location-mda-ip
>       Rule: score=-INFINITY boolean-op=or  (id:location-mda-ip-rule)
>         Expression: pingd lt 1  (id:location-mda-ip-rule-expr)
>         Expression: not_defined pingd  (id:location-mda-ip-rule-expr-1)

If I'm reading this right, you have two separate location constraints
for mda-ip, a positive preference for one particular node (score 50 for
MDA1PFP-PCS01), and a -INFINITY preference whenever the ping attribute
is bad.

> Ordering Constraints:
>   start ping-clone then start mda-ip (kind:Optional)
> (id:order-ping-clone-mda-ip-Optional)
>   promote drbd1_sync then start shared_fs (kind:Mandatory)
> (id:order-drbd1_sync-shared_fs-mandatory)
> Colocation Constraints:
>   ACTIVE with mda-ip (score:INFINITY) (id:colocation-ACTIVE-mda-ip-INFINITY)
>   drbd1_sync with mda-ip (score:INFINITY) (rsc-role:Master)
> (with-rsc-role:Started) (id:colocation-drbd1_sync-mda-ip-INFINITY)
>   shared_fs with drbd1_sync (score:INFINITY) (rsc-role:Started)
> (with-rsc-role:Master) (id:colocation-shared_fs-drbd1_sync-INFINITY)
> 
> As you can see I have defined a location constraint for the virtual ip
> mda-ip and colocated the drbd master and filesystem. However, this
> constraint is sometimes ignored. Can anybody point me to the correct
> configuration?

Which constraint is sometimes ignored?

> Is there a way to get more debugging output from pcs, e.g. what
> triggered actions, which scores are computed and from which values?

Not from pcs, but there are some lower-level tools that can sometimes be
helpful. "crm_simulate -sL" will show all the scores that went into the
current placement.

> 
> Best wishes,
>   Jens
> 
> --
> *Jens Auer *| CGI | Software-Engineer
> CGI (Germany) GmbH & Co. KG
> Rheinstraße 95 | 64295 Darmstadt | Germany
> T: +49 6151 36860 154
> _jens.auer at cgi.com_ <mailto:jens.auer at cgi.com>
> Unsere Pflichtangaben gemäß § 35a GmbHG / §§ 161, 125a HGB finden Sie
> unter _de.cgi.com/pflichtangaben_ <http://de.cgi.com/pflichtangaben>.