<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: Times New Roman; font-size: 12pt; color: #000000'><font size="3">Hi Jake,</font><div style="color: rgb(0, 0, 0); font-family: 'Times New Roman'; font-size: 12pt; "><br></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown" style="color: rgb(0, 0, 0); font-family: 'Times New Roman'; font-size: 12pt; ">Thanks, that helps to clarify how to properly create the colocation and order constraints and how the crm shell corresponds to the XML. You mentioned that I should "verify the statement didn't create a resource set and change the outcome a little" - can you elaborate on this a bit more? After I re-created the contraints, the XML lists two resource sets in each constraint:</div><div id=""><div id=""><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"><constraints></font></div><div id=""><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <rsc_order id="o_drbd-fs-vm" score="INFINITY"></font></div><div id=""><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_set action="promote" id="o_drbd-fs-vm-0"></font></div><div id=""><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_ref id="ms_drbd_r0"/></font></div><div id=""><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_ref id="ms_drbd_r1"/></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_ref id="ms_drbd_r2"/></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> </resource_set></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_set action="start" id="o_drbd-fs-vm-1"></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_ref id="p_fs_r0"/></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_ref id="p_virtualdomain_vm0"/></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> </resource_set></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> </rsc_order></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <rsc_colocation id="c_drbd_libvirt_vm" score="INFINITY"></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_set id="c_drbd_libvirt_vm-0" role="Master"></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_ref id="ms_drbd_r0"/></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_ref id="ms_drbd_r1"/></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_ref id="ms_drbd_r2"/></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> </resource_set></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2" id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"> <resource_set id="c_drbd_libvirt_vm-1"></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_ref id="p_fs_r0"/></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> <resource_ref id="p_virtualdomain_vm0"/></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> </resource_set></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"> </rsc_colocation></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><font face="'courier new', courier, monaco, monospace, sans-serif" size="2"></constraints></font></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><br></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown">Is this the desired behavior, or will having two resource sets cause the resources to not start linearly/in order as desired?</div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><br></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown">Thanks,</div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><br></div><div id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown">Andrew</div><br><hr id="zwchr" style="color: rgb(0, 0, 0); font-family: 'Times New Roman'; font-size: 12pt; "><div style="color: rgb(0, 0, 0); font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-size: 12pt; " id="aeaoofnhgocdbnbeljkmbjdmhbcokfdb-mousedown"><b>From: </b>"Jake Smith" <jsmith@argotec.com><br><b>To: </b>"The Pacemaker cluster resource manager" <pacemaker@oss.clusterlabs.org><br><b>Sent: </b>Friday, February 10, 2012 12:06:15 PM<br><b>Subject: </b>Re: [Pacemaker] Help with colocation and order of primitives<br><br><br>----- Original Message ----- <br><br>> From: "Andrew Martin" <amartin@xes-inc.com><br>> To: "The Pacemaker cluster resource manager"<br>> <pacemaker@oss.clusterlabs.org><br>> Sent: Friday, February 10, 2012 10:33:44 AM<br>> Subject: [Pacemaker] Help with colocation and order of primitives<br><br>> Hello,<br><br>> I am working on configuring a 2-node cluster with 3 DRBD devices,<br>> their corresponding filesystem mounts, and a VirtualDomain resource<br>> agent. I want to add the appropriate constraints to pacemaker so<br>> that these resources only start together on the same node, and that<br>> they start in the proper order. I have added the DRBD devices to a<br>> Master/Slave object, as outlined in<br>> http://www.linbit.com/fileadmin/tech-guides/ha-nfs.pdf . I would<br>> like them to start in the following order (and stop in the reverse<br>> order):<br>> ms_drbd_r0<br>> ms_drbd_r1<br>> ms_drbd_r2<br>> p_filesystem_r0<br>> p_filesystem_r1<br>> p_filesystem_r2<br>> p_virtualdomain_vm0<br><br>> I added the 3 filesystem primitives to a group, g_filesystems, which<br>> simplifies this somewhat. Are primitives in a group started in the<br>> order they are listed in a group, and stopped in the opposite order?<br><br>Yes and colocated in reverse...<br><br>group test A B C<br>order is A B C<br>colocation is C B A<br>Starts A then B then C, Stops C, B, A.<br><br>> I read through the Pacemaker Explained documentation on colocation<br>> and order,<br>> http://www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/s-resource-sets-ordering.html<br><br>It shows XML but when adding via the CRM CLI the rules work the same. The CRM CLI is just a(n easier) front-end for the XML (config is still written in XML)<br>Guide for CRM CLI is here:<br>http://www.clusterlabs.org/doc/crm_cli.html<br><br>> however it seems to mostly address the XML and not how to add<br>> constraints via the crm console (crm configure). It seems that I<br>> cannot add the Master/Slave objects to a group, so I will need to<br>> specify each of them individual in the constraints. Is it possible<br>> to include more than 2 resources to a constraint, e.g.<br>> colocation c_all_on_one inf: ms_drbd_r0:Master ms_drbd_r1:Master<br>> ms_drbd_r2:Master g_filesystems p_virtualdomain_vm0<br><br>Yes you can put more than one primitive in a colocation or order constraint - just be careful when adding multiples of different types of resource. You may want to do "crm configure show xml" to verify the statement didn't create a resource set and change the outcome a little.<br>Also with order statements make sure you specify the action when combining different types otherwise the first action will apply to all resources i.e. ms_drbd_r0:promote g_filesystems:start<br><br>> Or, can a single colocation or order constraint only contain 2<br>> resources? If so would I need to chain several constraints together,<br>> like this:<br>> colocation c_r0-r1 inf: ms_drbd_r0:Master ms_drbd_r1:Master<br>> colocation c_r1-r2 inf: ms_drbd_r1:Master ms_drbd:r2:Master<br>> colocation c_r2-fs inf: ms_drbd_r2:Master g_filesystems<br>> colocation c_fs_vm0 inf: g_filesystems p_virtualdomain_vm0<br><br>> Which syntax is supported and how can I achieve these colocation and<br>> order constraints?<br><br>> Thanks,<br><br>> Andrew<br>> _______________________________________________<br>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org<br>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker<br><br>> Project Home: http://www.clusterlabs.org<br>> Getting started:<br>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf<br>> Bugs: http://bugs.clusterlabs.org<br><br>_______________________________________________<br>Pacemaker mailing list: Pacemaker@oss.clusterlabs.org<br>http://oss.clusterlabs.org/mailman/listinfo/pacemaker<br><br>Project Home: http://www.clusterlabs.org<br>Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf<br>Bugs: http://bugs.clusterlabs.org<br></div><br></div></div></body></html>