[Pacemaker] Fw: Fw: Configuration for FS over DRBD over LVM
Andrew Beekhof
andrew at beekhof.net
Sun Jul 31 22:13:57 EDT 2011
On Thu, Jul 21, 2011 at 9:26 AM, Bob Schatz <bschatz at yahoo.com> wrote:
> Okay, this configuration works on one node (I am waiting for a hardware
> problem to be fixed before testing with second node):
> node cnode-1-3-5
> node cnode-1-3-6
> primitive glance-drbd ocf:linbit:drbd \
> params drbd_resource="glance-repos-drbd" \
> op start interval="0" timeout="240" \
> op stop interval="0" timeout="100" \
> op monitor interval="59s" role="Master" timeout="30s" \
> op monitor interval="61s" role="Slave" timeout="30s"
> primitive glance-fs ocf:heartbeat:Filesystem \
> params device="/dev/drbd1" directory="/glance-mount" fstype="ext4" \
> op start interval="0" timeout="60" \
> op monitor interval="60" timeout="60" OCF_CHECK_LEVEL="20" \
> op stop interval="0" timeout="120"
> primitive glance-ip ocf:heartbeat:IPaddr2 \
> params ip="10.4.0.25" nic="br100:1" \
> op monitor interval="5s"
> primitive glance-repos ocf:heartbeat:LVM \
> params volgrpname="glance-repos" exclusive="true" \
> op start interval="0" timeout="30" \
> op stop interval="0" timeout="30"
> group glance-repos-fs-group glance-fs glance-ip \
> meta target-role="Started"
> ms ms_drbd glance-drbd \
> meta master-node-max="1" clone-max="2" clone-node-max="1"
> globally-unique="false" notify="true" target-role="Master"
> colocation coloc-rule-w-master inf: ms_drbd:Master glance-repos-fs-group
> colocation coloc-rule-w-master2 inf: glance-repos ms_drbd:Master
> order glance-order-fs-after-drbd inf: glance-repos:start ms_drbd:start
This is the important one.
> order glance-order-fs-after-drbd-stop inf: glance-repos-fs-group:stop
> ms_drbd:demote
> order glance-order-fs-after-drbd-stop2 inf: ms_drbd:demote ms_drbd:stop
This one (above) is implied.
> order glance-order-fs-after-drbd-stop3 inf: ms_drbd:stop glance-repos:stop
> order glance-order-fs-after-drbd2 inf: ms_drbd:start ms_drbd:promote
As is this one.
> order glance-order-fs-after-drbd3 inf: ms_drbd:promote
> glance-repos-fs-group:start
> property $id="cib-bootstrap-options" \
> dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \
> cluster-infrastructure="openais" \
> expected-quorum-votes="1" \
> stonith-enabled="false" \
> no-quorum-policy="ignore" \
> last-lrm-refresh="1310768814"
> I will let everyone know how testing goes.
>
> Thanks,
> Bob
> ----- Forwarded Message -----
> From: Bob Schatz <bschatz at yahoo.com>
> To: "pacemaker at oss.clusterlabs.org" <pacemaker at oss.clusterlabs.org>
> Sent: Wednesday, July 20, 2011 1:38 PM
> Subject: [Pacemaker] Fw: Configuration for FS over DRBD over LVM
>
> One correction:
>
> I removed the "location" constraint and simply went with this:
> colocation coloc-rule-w-master inf: glance-repos ms_drbd:Master
> glance-repos-fs-group
> order glance-order-fs-after-drbd inf: glance-repos:start
> ms_drbd:promote glance-repos-fs-group:start
> order glance-order-fs-after-drbd2 inf: glance-repos-fs-group:stop
> ms_drbd:demote ms_drbd:stop glance-repos:stop
> I called out the stop of DRBD before the stop of LVM. The syslog attached
> previously is for this configuration.
>
> Thanks,
> Bob
> ________________________________
> From: Bob Schatz <bschatz at yahoo.com>
> To: "pacemaker at oss.clusterlabs.org" <pacemaker at oss.clusterlabs.org>
> Sent: Wednesday, July 20, 2011 11:32 AM
> Subject: [Pacemaker] Fw: Configuration for FS over DRBD over LVM
>
> I tried another test based on this thread:
> http://www.gossamer-threads.com/lists/linuxha/pacemaker/65928?search_string=lvm%20drbd;#65928
> I removed the "location" constraint and simply went with this:
> colocation coloc-rule-w-master inf: glance-repos ms_drbd:Master
> glance-repos-fs-group
> order glance-order-fs-after-drbd inf: glance-repos:start
> ms_drbd:promote glance-repos-fs-group:start
> order glance-order-fs-after-drbd2 inf: glance-repos-fs-group:stop
> ms_drbd:demote glance-repos:stop
> The stop actions were called in this order:
> stop file system
> demote DRBD
> stop LVM *****
> stop DRBD *****
> instead of:
> stop file system
> demote DRBD
> stop DRBD ******
> stop LVM ******
> I see these messages in the log which I believe are debug messages based on
> reading other threads:
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-0-start-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-0-start-end
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-0-stop-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-0-stop-end
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-1-promote-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-1-promote-end
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-1-demote-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-1-demote-end
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-2-start-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-2-start-end
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-2-stop-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd-2-stop-end
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-0-stop-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-0-stop-end
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-0-start-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-0-start-end
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-1-demote-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-1-demote-end
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-1-promote-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-1-promote-end
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-2-stop-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-2-stop-end
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-2-start-begin
> pengine: [21021]: debug: text2task: Unsupported action:
> glance-order-fs-after-drbd2-2-start-end
> I have attached a syslog-pacemaker log of the "/etc/init.d/corosync start"
> through "/etc/init.d/corosync stop" sequence.
>
> Thanks,
> Bob
> ----- Forwarded Message -----
> From: Bob Schatz <bschatz at yahoo.com>
> To: "pacemaker at oss.clusterlabs.org" <pacemaker at oss.clusterlabs.org>
> Sent: Tuesday, July 19, 2011 4:38 PM
> Subject: [Pacemaker] Configuration for FS over DRBD over LVM
>
> Hi,
> I am trying to configure an FS running on top of DRBD on top of LVM or:
> FS
> |
> DRBD
> |
> LVM
> I am using Pacemaker 1.0.8, Ubuntu 10.04 and DRBD 8.3.7.
> Reviewing all the manuals (Pacemaker Explained 1.0, DRBD 8.4 User Guide,
> etc) I came up with this Pacemaker configuration:
> node cnode-1-3-5
> node cnode-1-3-6
> primitive glance-drbd ocf:linbit:drbd \
> params drbd_resource="glance-repos-drbd" \
> op start interval="0" timeout="240" \
> op stop interval="0" timeout="100" \
> op monitor interval="59s" role="Master" timeout="30s" \
> op monitor interval="61s" role="Slave" timeout="30s"
> primitive glance-fs ocf:heartbeat:Filesystem \
> params device="/dev/drbd1" directory="/glance-mount" fstype="ext4" \
> op start interval="0" timeout="60" \
> op monitor interval="60" timeout="60" OCF_CHECK_LEVEL="20" \
> op stop interval="0" timeout="120"
> primitive glance-repos ocf:heartbeat:LVM \
> params volgrpname="glance-repos" exclusive="true" \
> op start interval="0" timeout="30" \
> op stop interval="0" timeout="30"
> group glance-repos-fs-group glance-fs
> ms ms_drbd glance-drbd \
> meta master-node-max="1" clone-max="2" clone-node-max="1"
> globally-unique="false" notify="true" target-role="Master"
> location drbd_on_node1 ms_drbd \
> rule $id="drbd_on_node1-rule" $role="Master" 100: #uname eq
> cnode-1-3-5
> colocation coloc-rule-w-master inf: glance-repos ms_drbd:Master
> order glance-order-fs-after-drbd inf: glance-repos:start ms_drbd:promote
> glance-repos-fs-group:start
> property $id="cib-bootstrap-options" \
> dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \
> cluster-infrastructure="openais" \
> expected-quorum-votes="1" \
> stonith-enabled="false" \
> no-quorum-policy="ignore" \
> last-lrm-refresh="1310768814"
> On one node, things come up cleanly. In fact, debug messages in the agent
> show that the start() functions for the agent are called and exited in order
> (LVM start, DRBD start and Filesystem start).
> The problem occurs when I do "/etc/init.d/corosync stop" on a single node.
> What happens is that the stop() functions are called in this order:
> 1. LVM stop
> 2. Filesystem stop
> 3. DRBD stop
> What I have tried:
> 1. I tried setting the score of the order to "500" assuming that this would
> mean the colocation rule would hit first. Still the same problem.
> 2. I tried leaving off the ":start" and ":promote" options on the "order"
> line. The stop order was still LVM, Filesystem, and DRBD
> 3. I tried adding another colocation rule "colocation coloc-rule-w-master2
> inf: ms_drbd:Master glance-repos-fs-group" to tie glance-repos-fs-group to
> the same node as DRBD. Stop still had the same issue. I assume that I
> will still need this rule when I add a second node to the test.
> Any suggestions would be appreciated.
> A side note, the reason I have a group for the file system is that I would
> like to add an application and IP address to the group once I get this
> working. Also, the reason I have LVM under DRBD is that I want to be able
> to grow the LVM volume as needed and then expand the DRBD volume.
>
> Thanks in advance,
> Bob
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs:
> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs:
> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs:
> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>
>
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs:
> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs:
> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs:
> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>
>
More information about the Pacemaker
mailing list