<div dir="ltr">Hello Ken,<div><br></div><div>here is the output of my other 2 clusters.</div><div><br></div><div><b>cluster 0:</b></div><div><div>Location Constraints:</div><div>Ordering Constraints:</div><div>Colocation Constraints:</div><div> virtual_ip with vg1 (score:INFINITY)</div><div> virtual_ip with lv1 (score:INFINITY)</div><div> virtual_ip with nfs-daemon (score:INFINITY)</div><div> virtual_ip with lv1_export (score:INFINITY)</div><div> virtual_ip with samba (score:INFINITY)</div><div> virtual_ip with lv1_export_squash (score:INFINITY)</div><div>Ticket Constraints:</div></div><div><br></div><div><b>and cluster 2:</b></div><div><div><div>Location Constraints:</div><div>Ordering Constraints:</div><div>Colocation Constraints:</div><div> lvm_scratch with cluster2_vip (score:INFINITY)</div><div> lun_scratch with cluster2_vip (score:INFINITY)</div><div> lvm_user_dirs with cluster2_vip (score:INFINITY)</div><div> lun_user_dirs with cluster2_vip (score:INFINITY)</div><div> lvm_user_dirs-bak with cluster2_vip (score:INFINITY)</div><div> nfsserver with cluster2_vip (score:INFINITY)</div><div> scratch-export-squash with cluster2_vip (score:INFINITY)</div><div> scratch-export-nosquash with cluster2_vip (score:INFINITY)</div><div> scratch-export-nosquash-linfarm with cluster2_vip (score:INFINITY)</div><div> scratch-export-nosquash-atlas with cluster2_vip (score:INFINITY)</div><div> user_dirs-export-squash with cluster2_vip (score:INFINITY)</div><div> user_dirs-export-nosquash with cluster2_vip (score:INFINITY)</div><div> user_dirs-export-nosquash-linfarm with cluster2_vip (score:INFINITY)</div><div> user_dirs-export-nosquash-atlas with cluster2_vip (score:INFINITY)</div><div> samba with cluster2_vip (score:INFINITY)</div><div> lvm_scratch_bak_btrfs with cluster2_vip (score:INFINITY)</div><div><br></div></div><div>As you can see, the output of cluster 0 is similar with cluster 1 but there are no messages for dependency loop. Shouldn't I have these errors on cluster 0 too?</div><div>Is cluster 2 output the one that you would recognize as correct?</div><div>Also, do you believe that these errors could be the source of the problem, that my resources return to the failed node, after it has resumed operation and do not stay on the one that took control of the resources after the failure?</div></div><div><br></div><div>Thanks for your time,</div><div>George.</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 26, 2018 at 11:01 PM, Ken Gaillot <span dir="ltr"><<a href="mailto:kgaillot@redhat.com" target="_blank">kgaillot@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On Fri, 2018-03-16 at 13:00 +0200, George Kourvoulis wrote:<br>
> Hi,<br>
><br>
> my logs keep being flooded by "Breaking dependency loop at<br>
> <rsc_name>" but I cannot figure out why. I haven't spotted such a<br>
> loop.<br>
><br>
> redhat-release CENTOS 7.2.1511<br>
> pcs --version 0.9.143<br>
><br>
> Here's an excerpt from the logs:<br>
><br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> lvm_titanas-bak<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> lvm_titanas-bak<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> lvm_titanas-bak<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
> Mar 16 07:29:54 [3670] filesrv12.localdomain pengine: info:<br>
> rsc_merge_weights: lvm_data-bak: Breaking dependency loop at<br>
> cluster1_vip<br>
><br>
> Here's the output of "pcs constraint --full"<br>
><br>
> Location Constraints:<br>
> Ordering Constraints:<br>
> Colocation Constraints:<br>
> cluster1_vip with lvm_data (score:INFINITY) (id:colocation-<br>
> cluster1_vip-lvm_data-<wbr>INFINITY)<br>
> cluster1_vip with lun_data (score:INFINITY) (id:colocation-<br>
> cluster1_vip-lun_data-<wbr>INFINITY)<br>
> cluster1_vip with lvm_titanas (score:INFINITY) (id:colocation-<br>
> cluster1_vip-lvm_titanas-<wbr>INFINITY)<br>
> cluster1_vip with lun_titanas (score:INFINITY) (id:colocation-<br>
> cluster1_vip-lun_titanas-<wbr>INFINITY)<br>
> cluster1_vip with lvm_misc (score:INFINITY) (id:colocation-<br>
> cluster1_vip-lvm_misc-<wbr>INFINITY)<br>
> cluster1_vip with lun_misc (score:INFINITY) (id:colocation-<br>
> cluster1_vip-lun_misc-<wbr>INFINITY)<br>
> cluster1_vip with lvm_data-bak (score:INFINITY) (id:colocation-<br>
> cluster1_vip-lvm_data-bak-<wbr>INFINITY)<br>
> cluster1_vip with lvm_titanas-bak (score:INFINITY) (id:colocation-<br>
> cluster1_vip-lvm_titanas-bak-<wbr>INFINITY)<br>
> cluster1_vip with lvm_misc-bak (score:INFINITY) (id:colocation-<br>
> cluster1_vip-lvm_misc-bak-<wbr>INFINITY)<br>
> cluster1_vip with nfsserver (score:INFINITY) (id:colocation-<br>
> cluster1_vip-nfsserver-<wbr>INFINITY)<br>
> cluster1_vip with data-export-squash (score:INFINITY)<br>
> (id:colocation-cluster1_vip-<wbr>data-export-squash-INFINITY)<br>
> cluster1_vip with data-export-nosquash (score:INFINITY)<br>
> (id:colocation-cluster1_vip-<wbr>data-export-nosquash-INFINITY)<br>
> cluster1_vip with data-export-nosquash-linfarm (score:INFINITY)<br>
> (id:colocation-cluster1_vip-<wbr>data-export-nosquash-linfarm-<wbr>INFINITY)<br>
> cluster1_vip with titanas-export-squash (score:INFINITY)<br>
> (id:colocation-cluster1_vip-<wbr>titanas-export-squash-<wbr>INFINITY)<br>
> cluster1_vip with titanas-export-nosquash (score:INFINITY)<br>
> (id:colocation-cluster1_vip-<wbr>titanas-export-nosquash-<wbr>INFINITY)<br>
> cluster1_vip with titanas-export-nosquash-<wbr>linfarm (score:INFINITY)<br>
> (id:colocation-cluster1_vip-<wbr>titanas-export-nosquash-<wbr>linfarm-INFINITY)<br>
> cluster1_vip with misc-export-squash (score:INFINITY)<br>
> (id:colocation-cluster1_vip-<wbr>misc-export-squash-INFINITY)<br>
> cluster1_vip with misc-export-nosquash (score:INFINITY)<br>
> (id:colocation-cluster1_vip-<wbr>misc-export-nosquash-INFINITY)<br>
> cluster1_vip with misc-export-nosquash-linfarm (score:INFINITY)<br>
> (id:colocation-cluster1_vip-<wbr>misc-export-nosquash-linfarm-<wbr>INFINITY)<br>
> cluster1_vip with samba (score:INFINITY) (id:colocation-<br>
> cluster1_vip-samba-INFINITY)<br>
> cluster1_vip with httpd (score:INFINITY) (id:colocation-<br>
> cluster1_vip-httpd-INFINITY)<br>
<br>
</div></div>I'm guessing you have the sense of colocation constraints flipped. The<br>
above says: place a whole bunch of resources first, with no<br>
dependencies among them, then try to place cluster1_vip on a node with<br>
all of them (which is likely impossible).<br>
<br>
If you want to pick a node for the IP first, then place all the<br>
resources on that node, flip the constraints, e.g. httpd with<br>
cluster1_vip.<br>
<span class=""><br>
><br>
> Here's my pcs cluster cib output:<br>
> <a href="https://pastebin.com/CkW5wQmS" rel="noreferrer" target="_blank">https://pastebin.com/CkW5wQmS</a><br>
><br>
> I am afraid that this could be the cause of my resources falling back<br>
> to a node that has recovered from a fail over although I have a<br>
> stickiness score of INFINITY.<br>
><br>
> Thanks,<br>
> George<br>
</span>> ______________________________<wbr>_________________<br>
> Users mailing list: <a href="mailto:Users@clusterlabs.org">Users@clusterlabs.org</a><br>
> <a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.clusterlabs.org/<wbr>mailman/listinfo/users</a><br>
><br>
> Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
> Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch" rel="noreferrer" target="_blank">http://www.clusterlabs.org/<wbr>doc/Cluster_from_Scratch</a>.<br>
> pdf<br>
> Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
<span class="HOEnZb"><font color="#888888">--<br>
Ken Gaillot <<a href="mailto:kgaillot@redhat.com">kgaillot@redhat.com</a>><br>
______________________________<wbr>_________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org">Users@clusterlabs.org</a><br>
<a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.clusterlabs.org/<wbr>mailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">http://www.clusterlabs.org/<wbr>doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
</font></span></blockquote></div><br></div>