[ClusterLabs] Dependency loop

[Ken Gaillot]

On Fri, 2018-03-16 at 13:00 +0200, George Kourvoulis wrote:
> Hi,
> 
> my logs keep being flooded by "Breaking dependency loop at
> <rsc_name>" but I cannot figure out why. I haven't spotted such a
> loop.
> 
> redhat-release CENTOS 7.2.1511
> pcs --version 0.9.143
> 
> Here's an excerpt from the logs:
> 
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> lvm_titanas-bak
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> lvm_titanas-bak
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> lvm_titanas-bak
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> Mar 16 07:29:54 [3670] filesrv12.localdomain    pengine:     info:
> rsc_merge_weights:	lvm_data-bak: Breaking dependency loop at
> cluster1_vip
> 
> Here's the output of "pcs constraint --full"
> 
> Location Constraints:
> Ordering Constraints:
> Colocation Constraints:
>   cluster1_vip with lvm_data (score:INFINITY) (id:colocation-
> cluster1_vip-lvm_data-INFINITY)
>   cluster1_vip with lun_data (score:INFINITY) (id:colocation-
> cluster1_vip-lun_data-INFINITY)
>   cluster1_vip with lvm_titanas (score:INFINITY) (id:colocation-
> cluster1_vip-lvm_titanas-INFINITY)
>   cluster1_vip with lun_titanas (score:INFINITY) (id:colocation-
> cluster1_vip-lun_titanas-INFINITY)
>   cluster1_vip with lvm_misc (score:INFINITY) (id:colocation-
> cluster1_vip-lvm_misc-INFINITY)
>   cluster1_vip with lun_misc (score:INFINITY) (id:colocation-
> cluster1_vip-lun_misc-INFINITY)
>   cluster1_vip with lvm_data-bak (score:INFINITY) (id:colocation-
> cluster1_vip-lvm_data-bak-INFINITY)
>   cluster1_vip with lvm_titanas-bak (score:INFINITY) (id:colocation-
> cluster1_vip-lvm_titanas-bak-INFINITY)
>   cluster1_vip with lvm_misc-bak (score:INFINITY) (id:colocation-
> cluster1_vip-lvm_misc-bak-INFINITY)
>   cluster1_vip with nfsserver (score:INFINITY) (id:colocation-
> cluster1_vip-nfsserver-INFINITY)
>   cluster1_vip with data-export-squash (score:INFINITY)
> (id:colocation-cluster1_vip-data-export-squash-INFINITY)
>   cluster1_vip with data-export-nosquash (score:INFINITY)
> (id:colocation-cluster1_vip-data-export-nosquash-INFINITY)
>   cluster1_vip with data-export-nosquash-linfarm (score:INFINITY)
> (id:colocation-cluster1_vip-data-export-nosquash-linfarm-INFINITY)
>   cluster1_vip with titanas-export-squash (score:INFINITY)
> (id:colocation-cluster1_vip-titanas-export-squash-INFINITY)
>   cluster1_vip with titanas-export-nosquash (score:INFINITY)
> (id:colocation-cluster1_vip-titanas-export-nosquash-INFINITY)
>   cluster1_vip with titanas-export-nosquash-linfarm (score:INFINITY)
> (id:colocation-cluster1_vip-titanas-export-nosquash-linfarm-INFINITY)
>   cluster1_vip with misc-export-squash (score:INFINITY)
> (id:colocation-cluster1_vip-misc-export-squash-INFINITY)
>   cluster1_vip with misc-export-nosquash (score:INFINITY)
> (id:colocation-cluster1_vip-misc-export-nosquash-INFINITY)
>   cluster1_vip with misc-export-nosquash-linfarm (score:INFINITY)
> (id:colocation-cluster1_vip-misc-export-nosquash-linfarm-INFINITY)
>   cluster1_vip with samba (score:INFINITY) (id:colocation-
> cluster1_vip-samba-INFINITY)
>   cluster1_vip with httpd (score:INFINITY) (id:colocation-
> cluster1_vip-httpd-INFINITY)

I'm guessing you have the sense of colocation constraints flipped. The
above says: place a whole bunch of resources first, with no
dependencies among them, then try to place cluster1_vip on a node with
all of them (which is likely impossible).

If you want to pick a node for the IP first, then place all the
resources on that node, flip the constraints, e.g. httpd with
cluster1_vip.

> 
> Here's my pcs cluster cib output:
> https://pastebin.com/CkW5wQmS
> 
> I am afraid that this could be the cause of my resources falling back
> to a node that has recovered from a fail over although I have a
> stickiness score of INFINITY.
> 
> Thanks,
> George
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> https://lists.clusterlabs.org/mailman/listinfo/users
> 
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.
> pdf
> Bugs: http://bugs.clusterlabs.org
-- 
Ken Gaillot <kgaillot at redhat.com>