<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">Hi,</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">I am trying to configure an FS running on top of DRBD on top of LVM or:</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "> FS</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "> |</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "> DRBD</div><div style="font-family: 'times
new roman', 'new york', times, serif; font-size: 12pt; "> |</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "> LVM</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">I am using Pacemaker 1.0.8, Ubuntu 10.04 and DRBD 8.3.7.</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">Reviewing all the manuals (Pacemaker Explained 1.0, DRBD 8.4 User Guide, etc) I came up with this Pacemaker configuration:</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span
class="Apple-tab-span" style="white-space:pre"> </span>node cnode-1-3-5</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span>node cnode-1-3-6</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span>primitive glance-drbd ocf:linbit:drbd \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> params drbd_resource="glance-repos-drbd" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> op start interval="0" timeout="240" \</font></div><div><font class="Apple-style-span" face="'times
new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> op stop interval="0" timeout="100" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> op monitor interval="59s" role="Master" timeout="30s" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> op monitor interval="61s" role="Slave" timeout="30s"</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span>primitive glance-fs ocf:heartbeat:Filesystem \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"> <span
class="Apple-tab-span" style="white-space:pre"> </span> params device="/dev/drbd1" directory="/glance-mount" fstype="ext4" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"> <span class="Apple-tab-span" style="white-space:pre"> </span> op start interval="0" timeout="60" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"> <span class="Apple-tab-span" style="white-space:pre"> </span> op monitor interval="60" timeout="60" OCF_CHECK_LEVEL="20" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"> <span class="Apple-tab-span" style="white-space:pre"> </span> op stop interval="0" timeout="120"</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span
class="Apple-tab-span" style="white-space:pre"> </span>primitive glance-repos ocf:heartbeat:LVM \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> params volgrpname="glance-repos" exclusive="true" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> op start interval="0" timeout="30" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> op stop interval="0" timeout="30"</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span>group
glance-repos-fs-group glance-fs</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span>ms ms_drbd glance-drbd \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> meta master-node-max="1" clone-max="2" clone-node-max="1" globally-unique="false" notify="true" target-role="Master"</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span>location drbd_on_node1 ms_drbd \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> rule $id="drbd_on_node1-rule" $role="Master" 100: #uname eq
cnode-1-3-5</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span>colocation coloc-rule-w-master inf: glance-repos ms_drbd:Master</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span>order glance-order-fs-after-drbd inf: glance-repos:start ms_drbd:promote glance-repos-fs-group:start</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span>property $id="cib-bootstrap-options" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd"
\</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> cluster-infrastructure="openais" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> expected-quorum-votes="1" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> stonith-enabled="false" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times, serif"><span class="Apple-tab-span" style="white-space:pre"> </span> no-quorum-policy="ignore" \</font></div><div><font class="Apple-style-span" face="'times new roman', 'new york', times,
serif"><span class="Apple-tab-span" style="white-space:pre"> </span> last-lrm-refresh="1310768814"</font></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">On one node, things come up cleanly. In fact, debug messages in the agent show that the start() functions for the agent are called and exited in order (LVM start, DRBD start and Filesystem start).</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">The problem occurs when I do "/etc/init.d/corosync stop" on a single node. What happens is that the stop() functions are called in this order:</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt;
"><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><span class="Apple-tab-span" style="white-space:pre"> </span>1. LVM stop</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><span class="Apple-tab-span" style="white-space:pre"> </span>2. Filesystem stop</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><span class="Apple-tab-span" style="white-space:pre"> </span>3. DRBD stop</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">What I have tried:</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><span class="Apple-tab-span"
style="white-space:pre"> </span>1. I tried setting the score of the order to "500" assuming that this would mean the colocation rule would hit first. Still the same problem.</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><span class="Apple-tab-span" style="white-space:pre"> </span>2. I tried leaving off the ":start" and ":promote" options on the "order" line. The stop order was still LVM, Filesystem, and DRBD</div><div><span class="Apple-tab-span" style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; white-space: pre; "> </span><font class="Apple-style-span" face="'times new roman', 'new york', times, serif" size="3">3. I tried adding another colocation rule "</font><font class="Apple-style-span" face="'times new roman', 'new york', times, serif">colocation coloc-rule-w-master2 inf: ms_drbd:Master glance-repos-fs-group" to tie glance-repos-fs-group to the same
node as DRBD. Stop still had the same issue. I assume that I will still need this rule when I add a second node to the test.</font></div><div><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">Any suggestions would be appreciated.</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">A side note, the reason I have a group for the file system is that I would like to add an application and IP address to the group once I get this working. Also, the reason I have LVM under DRBD is that I want to be able to grow the LVM volume as needed and then expand the DRBD volume.</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size:
12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">Thanks in advance,</div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; "><br></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt; ">Bob</div></div></div></body></html>