<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>I had seemed to cause a split brain attempting to repair this. But that wasn't the issue. You can't have any colocation requirements for DRBD resources; that's what killed me. This line did it:</div>
<div><br>
</div>
<div> ms_drbd_vmfs with ClusterIP (score:INFINITY) (id:colocation-ms_drbd_vmfs-ClusterIP-INFINITY)</div>
<div><br>
</div>
<div>Do NOT do this!</div>
<div><br>
</div>
<div>Jason</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Jason Gress <<a href="mailto:jgress@accertify.com">jgress@accertify.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org">users@clusterlabs.org</a>><br>
<span style="font-weight:bold">Date: </span>Friday, September 18, 2015 at 3:03 PM<br>
<span style="font-weight:bold">To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org">users@clusterlabs.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [ClusterLabs] Pacemaker/pcs & DRBD not demoting secondary node to Slave (always Stopped)<br>
</div>
<div><br>
</div>
<div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>Well, it almost worked. I was able to modify the existing cluster per your command, and it worked great.</div>
<div><br>
</div>
<div>Today, I made two more clusters via the exact same process (I used/modified my notes as I was building and fixing the first one yesterday) and now it's doing the same thing, despite having your improved master slave rule. Here's the config:</div>
<div><br>
</div>
<div>
<div>[root@fx201-1a ~]# pcs config --full</div>
<div>Cluster Name: fx201-vmcl</div>
<div>Corosync Nodes:</div>
<div> fx201-1a.zwo fx201-1b.zwo</div>
<div>Pacemaker Nodes:</div>
<div> fx201-1a.zwo fx201-1b.zwo</div>
<div><br>
</div>
<div>Resources:</div>
<div> Resource: ClusterIP (class=ocf provider=heartbeat type=IPaddr2)</div>
<div> Attributes: ip=10.XX.XX.XX cidr_netmask=24</div>
<div> Operations: start interval=0s timeout=20s (ClusterIP-start-timeout-20s)</div>
<div> stop interval=0s timeout=20s (ClusterIP-stop-timeout-20s)</div>
<div> monitor interval=15s (ClusterIP-monitor-interval-15s)</div>
<div> Master: ms_drbd_vmfs</div>
<div> Meta Attrs: master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true</div>
<div> Resource: drbd_vmfs (class=ocf provider=linbit type=drbd)</div>
<div> Attributes: drbd_resource=vmfs</div>
<div> Operations: start interval=0s timeout=240 (drbd_vmfs-start-timeout-240)</div>
<div> promote interval=0s timeout=90 (drbd_vmfs-promote-timeout-90)</div>
<div> demote interval=0s timeout=90 (drbd_vmfs-demote-timeout-90)</div>
<div> stop interval=0s timeout=100 (drbd_vmfs-stop-timeout-100)</div>
<div> monitor interval=29s role=Master (drbd_vmfs-monitor-interval-29s-role-Master)</div>
<div> monitor interval=31s role=Slave (drbd_vmfs-monitor-interval-31s-role-Slave)</div>
<div> Resource: vmfsFS (class=ocf provider=heartbeat type=Filesystem)</div>
<div> Attributes: device=/dev/drbd0 directory=/exports/vmfs fstype=xfs</div>
<div> Operations: start interval=0s timeout=60 (vmfsFS-start-timeout-60)</div>
<div> stop interval=0s timeout=60 (vmfsFS-stop-timeout-60)</div>
<div> monitor interval=20 timeout=40 (vmfsFS-monitor-interval-20)</div>
<div> Resource: nfs-server (class=systemd type=nfs-server)</div>
<div> Operations: monitor interval=60s (nfs-server-monitor-interval-60s)</div>
<div><br>
</div>
<div>Stonith Devices:</div>
<div>Fencing Levels:</div>
<div><br>
</div>
<div>Location Constraints:</div>
<div>Ordering Constraints:</div>
<div> promote ms_drbd_vmfs then start vmfsFS (kind:Mandatory) (id:order-ms_drbd_vmfs-vmfsFS-mandatory)</div>
<div> start vmfsFS then start nfs-server (kind:Mandatory) (id:order-vmfsFS-nfs-server-mandatory)</div>
<div> start ClusterIP then start nfs-server (kind:Mandatory) (id:order-ClusterIP-nfs-server-mandatory)</div>
<div>Colocation Constraints:</div>
<div> ms_drbd_vmfs with ClusterIP (score:INFINITY) (id:colocation-ms_drbd_vmfs-ClusterIP-INFINITY)</div>
<div> vmfsFS with ms_drbd_vmfs (score:INFINITY) (with-rsc-role:Master) (id:colocation-vmfsFS-ms_drbd_vmfs-INFINITY)</div>
<div> nfs-server with vmfsFS (score:INFINITY) (id:colocation-nfs-server-vmfsFS-INFINITY)</div>
<div> nfs-server with ClusterIP (score:INFINITY) (id:colocation-nfs-server-ClusterIP-INFINITY)</div>
<div><br>
</div>
<div>Cluster Properties:</div>
<div> cluster-infrastructure: corosync</div>
<div> cluster-name: fx201-vmcl</div>
<div> dc-version: 1.1.13-a14efad</div>
<div> have-watchdog: false</div>
<div> stonith-enabled: false</div>
</div>
<div><br>
</div>
<div>
<div>[root@fx201-1a ~]# pcs status --full</div>
<div>Cluster name: fx201-vmcl</div>
<div>Last updated: Fri Sep 18 15:02:16 2015<span class="Apple-tab-span" style="white-space:pre"></span>Last change: Fri Sep 18 14:44:33 2015 by root via crm_attribute on fx201-1b.zwo</div>
<div>Stack: corosync</div>
<div>Current DC: fx201-1a.zwo (1) (version 1.1.13-a14efad) - partition with quorum</div>
<div>2 nodes and 5 resources configured</div>
<div><br>
</div>
<div>Online: [ fx201-1a.zwo (1) fx201-1b.zwo (2) ]</div>
<div><br>
</div>
<div>Full list of resources:</div>
<div><br>
</div>
<div> ClusterIP<span class="Apple-tab-span" style="white-space:pre"> </span>(ocf::heartbeat:IPaddr2):<span class="Apple-tab-span" style="white-space:pre"></span>Started fx201-1a.zwo</div>
<div> Master/Slave Set: ms_drbd_vmfs [drbd_vmfs]</div>
<div> drbd_vmfs<span class="Apple-tab-span" style="white-space:pre"> </span>(ocf::linbit:drbd):<span class="Apple-tab-span" style="white-space:pre"></span>Master fx201-1a.zwo</div>
<div> drbd_vmfs<span class="Apple-tab-span" style="white-space:pre"> </span>(ocf::linbit:drbd):<span class="Apple-tab-span" style="white-space:pre"></span>Stopped</div>
<div> Masters: [ fx201-1a.zwo ]</div>
<div> Stopped: [ fx201-1b.zwo ]</div>
<div> vmfsFS<span class="Apple-tab-span" style="white-space:pre"> </span>(ocf::heartbeat:Filesystem):<span class="Apple-tab-span" style="white-space:pre"></span>Started fx201-1a.zwo</div>
<div> nfs-server<span class="Apple-tab-span" style="white-space:pre"> </span>(systemd:nfs-server):<span class="Apple-tab-span" style="white-space:pre"></span>Started fx201-1a.zwo</div>
<div><br>
</div>
<div>PCSD Status:</div>
<div> fx201-1a.zwo: Online</div>
<div> fx201-1b.zwo: Online</div>
<div><br>
</div>
<div>Daemon Status:</div>
<div> corosync: active/enabled</div>
<div> pacemaker: active/enabled</div>
<div> pcsd: active/enabled</div>
</div>
<div><br>
</div>
<div>This is so strange... The master/slave rule fixed my other two clusters, but not this one.</div>
<div><br>
</div>
<div>Thank you all for your advice,</div>
<div><br>
</div>
<div>Jason</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Jason Gress <<a href="mailto:jgress@accertify.com">jgress@accertify.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org">users@clusterlabs.org</a>><br>
<span style="font-weight:bold">Date: </span>Thursday, September 17, 2015 at 7:25 PM<br>
<span style="font-weight:bold">To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org">users@clusterlabs.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [ClusterLabs] Pacemaker/pcs & DRBD not demoting secondary node to Slave (always Stopped)<br>
</div>
<div><br>
</div>
<div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>That was Exactly what I needed. Thank you so much!</div>
<div><br>
</div>
<div>Jason</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Luke Pascoe <<a href="mailto:luke@osnz.co.nz">luke@osnz.co.nz</a>><br>
<span style="font-weight:bold">Reply-To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org">users@clusterlabs.org</a>><br>
<span style="font-weight:bold">Date: </span>Thursday, September 17, 2015 at 7:08 PM<br>
<span style="font-weight:bold">To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org">users@clusterlabs.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [ClusterLabs] Pacemaker/pcs & DRBD not demoting secondary node to Slave (always Stopped)<br>
</div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">pcs resource create drbd_iscsivg0 ocf:linbit:drbd drbd_resource=iscsivg0 op monitor interval="29s" role="Master" op monitor interval="31s" role="Slave"<br>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div class="gmail_signature">
<p style="font-family:verdana,sans-serif"><span style="font-weight:bold">Luke Pasc</span><span style="font-weight:bold">oe</span></p>
<p style="font-family:verdana,sans-serif"><img src="http://osnz.co.nz/logo_blue_80.png" width="96" height="28"><font size="1"><br>
<b><br>
</b></font></p>
<p style="font-family:verdana,sans-serif"><font size="1"><b>E</b> <a href="mailto:luke@osnz.co.nz" target="_blank">
luke@osnz.co.nz</a><br>
<b>P</b> +64 (9) 296 2961<br>
<b>M</b> +64 (27) 426 6649<br>
<b>W</b> <a href="http://www.osnz.co.nz/" target="_blank">www.osnz.co.nz</a><br>
<br>
24 Wellington St<br>
Papakura<br>
Auckland, 2110 <br>
New Zealand</font></p>
</div>
</div>
<br>
<div class="gmail_quote">On 18 September 2015 at 12:02, Jason Gress <span dir="ltr">
<<a href="mailto:jgress@accertify.com" target="_blank">jgress@accertify.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>That may very well be it. Would you be so kind as to show me the pcs command to create that config? I generated my configuration with these commands, and I'm not sure how to get the additional monitor options in there:</div>
<div><br>
</div>
<div>
<div>
<div style="font-size:13.3333px;font-family:monospace">pcs resource create drbd_vmfs ocf:linbit:drbd <span style="font-size:13.3333px">drbd_resource=vmfs op monitor interval=30s</span></div>
</div>
<div style="font-size:13.3333px;font-family:monospace">
<div style="font-size:13.3333px">pcs resource master ms_drbd_vmfs drbd_vmfs <span style="font-size:13.3333px">master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 </span><span style="font-size:13.3333px">notify=true</span></div>
<div style="font-size:13.3333px"><span style="font-size:13.3333px"><br>
</span></div>
<div style="font-size:13.3333px"><span style="font-size:13.3333px">Thank you very much for your help, and sorry for the newbie question!</span></div>
<div style="font-size:13.3333px"><span style="font-size:13.3333px"><br>
</span></div>
<div style="font-size:13.3333px"><span style="font-size:13.3333px">Jason</span></div>
</div>
</div>
<div><br>
</div>
<span>
<div style="font-family:Calibri;font-size:11pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt">
<span class=""><span style="font-weight:bold">From: </span>Luke Pascoe <<a href="mailto:luke@osnz.co.nz" target="_blank">luke@osnz.co.nz</a>><br>
<span style="font-weight:bold">Reply-To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org" target="_blank">users@clusterlabs.org</a>><br>
</span><span style="font-weight:bold">Date: </span>Thursday, September 17, 2015 at 6:54 PM
<div>
<div class="h5"><br>
<span style="font-weight:bold">To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org" target="_blank">users@clusterlabs.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [ClusterLabs] Pacemaker/pcs & DRBD not demoting secondary node to Slave (always Stopped)<br>
</div>
</div>
</div>
<div>
<div class="h5">
<div><br>
</div>
<div>
<div>
<div dir="ltr">The only difference in the DRBD resource between yours and mine that I can see is the monitoring parameters (mine works nicely, but is Centos 6). Here's mine:
<div><br>
</div>
<div>
<div>Master: ms_drbd_iscsicg0</div>
<div> Meta Attrs: master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true </div>
<div> Resource: drbd_iscsivg0 (class=ocf provider=linbit type=drbd)</div>
<div> Attributes: drbd_resource=iscsivg0 </div>
<div> Operations: start interval=0s timeout=240 (drbd_iscsivg0-start-timeout-240)</div>
<div> promote interval=0s timeout=90 (drbd_iscsivg0-promote-timeout-90)</div>
<div> demote interval=0s timeout=90 (drbd_iscsivg0-demote-timeout-90)</div>
<div> stop interval=0s timeout=100 (drbd_iscsivg0-stop-timeout-100)</div>
<div> monitor interval=29s role=Master (drbd_iscsivg0-monitor-interval-29s-role-Master)</div>
<div> monitor interval=31s role=Slave (drbd_iscsivg0-monitor-interval-31s-role-Slave)</div>
</div>
<div><br>
</div>
<div>What mechanism are you using to fail over? Check your constraints after you do it and make sure it hasn't added one which stops the slave clone from starting on the "failed" node.</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div>
<p style="font-family:verdana,sans-serif"><span style="font-weight:bold">Luke Pasc</span><span style="font-weight:bold">oe</span></p>
<p style="font-family:verdana,sans-serif"><img src="http://osnz.co.nz/logo_blue_80.png" width="96" height="28"><font size="1"><br>
<b><br>
</b></font></p>
<p style="font-family:verdana,sans-serif"><font size="1"><b>E</b> <a href="mailto:luke@osnz.co.nz" target="_blank">
luke@osnz.co.nz</a><br>
<b>P</b> <a href="tel:%2B64%20%289%29%20296%202961" value="+6492962961" target="_blank">
+64 (9) 296 2961</a><br>
<b>M</b> +64 (27) 426 6649<br>
<b>W</b> <a href="http://www.osnz.co.nz/" target="_blank">www.osnz.co.nz</a><br>
<br>
24 Wellington St<br>
Papakura<br>
Auckland, 2110 <br>
New Zealand</font></p>
</div>
</div>
<br>
<div class="gmail_quote">On 18 September 2015 at 11:40, Jason Gress <span dir="ltr">
<<a href="mailto:jgress@accertify.com" target="_blank">jgress@accertify.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>Looking more closely, according to page 64 (<a href="http://clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://clusterlabs.org/doc/Cluster_from_Scratch.pdf</a>) it does indeed appear that 1 is the correct number. (I just realized that
it's page 64 of the "book", but page 76 of the pdf.)</div>
<div><br>
</div>
<div>Thank you again,</div>
<div><br>
</div>
<div>Jason</div>
<div><br>
</div>
<span>
<div style="font-family:Calibri;font-size:11pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt">
<span style="font-weight:bold">From: </span>Jason Gress <<a href="mailto:jgress@accertify.com" target="_blank">jgress@accertify.com</a>><span><br>
<span style="font-weight:bold">Reply-To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org" target="_blank">users@clusterlabs.org</a>><br>
</span><span style="font-weight:bold">Date: </span>Thursday, September 17, 2015 at 6:36 PM<span><br>
<span style="font-weight:bold">To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org" target="_blank">users@clusterlabs.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [ClusterLabs] Pacemaker/pcs & DRBD not demoting secondary node to Slave (always Stopped)<br>
</span></div>
<div><br>
</div>
<div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>I can't say whether or not you are right or wrong (you may be right!) but I followed the Cluster From Scratch tutorial closely, and it only had a clone-node-max=1 there. (Page 106 of the pdf, for the curious.)</div>
<div><br>
</div>
<div>Thanks,</div>
<div><br>
</div>
<div>Jason</div>
<div><br>
</div>
<span>
<div>
<div>
<div style="font-family:Calibri;font-size:11pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt">
<span style="font-weight:bold">From: </span>Luke Pascoe <<a href="mailto:luke@osnz.co.nz" target="_blank">luke@osnz.co.nz</a>><br>
<span style="font-weight:bold">Reply-To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org" target="_blank">users@clusterlabs.org</a>><br>
<span style="font-weight:bold">Date: </span>Thursday, September 17, 2015 at 6:29 PM<br>
<span style="font-weight:bold">To: </span>Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org" target="_blank">users@clusterlabs.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [ClusterLabs] Pacemaker/pcs & DRBD not demoting secondary node to Slave (always Stopped)<br>
</div>
<div><br>
</div>
</div>
</div>
<div>
<div>
<div>
<div>
<div dir="ltr">I may be wrong, but shouldn't "<span style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">clone-node-max" be 2 on the </span><span style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">ms_drbd_vmfs
resource?</span></div>
</div>
</div>
<div class="gmail_extra">
<div>
<div><br clear="all">
<div>
<div>
<p style="font-family:verdana,sans-serif"><span style="font-weight:bold">Luke Pasc</span><span style="font-weight:bold">oe</span></p>
<p style="font-family:verdana,sans-serif"><img src="http://osnz.co.nz/logo_blue_80.png" width="96" height="28"><font size="1"><br>
<b><br>
</b></font></p>
<p style="font-family:verdana,sans-serif"><font size="1"><b>E</b> <a href="mailto:luke@osnz.co.nz" target="_blank">
luke@osnz.co.nz</a><br>
<b>P</b> <a href="tel:%2B64%20%289%29%20296%202961" value="+6492962961" target="_blank">
+64 (9) 296 2961</a><br>
<b>M</b> +64 (27) 426 6649<br>
<b>W</b> <a href="http://www.osnz.co.nz/" target="_blank">www.osnz.co.nz</a><br>
<br>
24 Wellington St<br>
Papakura<br>
Auckland, 2110 <br>
New Zealand</font></p>
</div>
</div>
<br>
</div>
</div>
<div class="gmail_quote">
<div>
<div>On 18 September 2015 at 11:02, Jason Gress <span dir="ltr"><<a href="mailto:jgress@accertify.com" target="_blank">jgress@accertify.com</a>></span> wrote:<br>
</div>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>
<div>
<div>I have a simple DRBD + filesystem + NFS configuration that works properly when I manually start/stop DRBD, but will not start the DRBD slave resource properly on failover or recovery. I cannot ever get the Master/Slave set to say anything but 'Stopped'.
I am running CentOS 7.1 with the latest packages as of today:</div>
<div><br>
</div>
<div>
<div>[root@fx201-1a log]# rpm -qa | grep -e pcs -e pacemaker -e drbd</div>
<div>pacemaker-cluster-libs-1.1.12-22.el7_1.4.x86_64</div>
<div>pacemaker-1.1.12-22.el7_1.4.x86_64</div>
<div>pcs-0.9.137-13.el7_1.4.x86_64</div>
<div>pacemaker-libs-1.1.12-22.el7_1.4.x86_64</div>
<div>drbd84-utils-8.9.3-1.1.el7.elrepo.x86_64</div>
<div>pacemaker-cli-1.1.12-22.el7_1.4.x86_64</div>
<div>kmod-drbd84-8.4.6-1.el7.elrepo.x86_64</div>
</div>
<div><br>
</div>
<div>Here is my pcs config output:</div>
<div><br>
</div>
<div>
<div>[root@fx201-1a log]# pcs config</div>
<div>Cluster Name: fx201-vmcl</div>
<div>Corosync Nodes:</div>
<div> fx201-1a.ams fx201-1b.ams</div>
<div>Pacemaker Nodes:</div>
<div> fx201-1a.ams fx201-1b.ams</div>
<div><br>
</div>
<div>Resources:</div>
<div> Resource: ClusterIP (class=ocf provider=heartbeat type=IPaddr2)</div>
<div> Attributes: ip=10.XX.XX.XX cidr_netmask=24</div>
<div> Operations: start interval=0s timeout=20s (ClusterIP-start-timeout-20s)</div>
<div> stop interval=0s timeout=20s (ClusterIP-stop-timeout-20s)</div>
<div> monitor interval=15s (ClusterIP-monitor-interval-15s)</div>
<div> Master: ms_drbd_vmfs</div>
<div> Meta Attrs: master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true</div>
<div> Resource: drbd_vmfs (class=ocf provider=linbit type=drbd)</div>
<div> Attributes: drbd_resource=vmfs</div>
<div> Operations: start interval=0s timeout=240 (drbd_vmfs-start-timeout-240)</div>
<div> promote interval=0s timeout=90 (drbd_vmfs-promote-timeout-90)</div>
<div> demote interval=0s timeout=90 (drbd_vmfs-demote-timeout-90)</div>
<div> stop interval=0s timeout=100 (drbd_vmfs-stop-timeout-100)</div>
<div> monitor interval=30s (drbd_vmfs-monitor-interval-30s)</div>
<div> Resource: vmfsFS (class=ocf provider=heartbeat type=Filesystem)</div>
<div> Attributes: device=/dev/drbd0 directory=/exports/vmfs fstype=xfs</div>
<div> Operations: start interval=0s timeout=60 (vmfsFS-start-timeout-60)</div>
<div> stop interval=0s timeout=60 (vmfsFS-stop-timeout-60)</div>
<div> monitor interval=20 timeout=40 (vmfsFS-monitor-interval-20)</div>
<div> Resource: nfs-server (class=systemd type=nfs-server)</div>
<div> Operations: monitor interval=60s (nfs-server-monitor-interval-60s)</div>
<div><br>
</div>
<div>Stonith Devices:</div>
<div>Fencing Levels:</div>
<div><br>
</div>
<div>Location Constraints:</div>
<div>Ordering Constraints:</div>
<div> promote ms_drbd_vmfs then start vmfsFS (kind:Mandatory) (id:order-ms_drbd_vmfs-vmfsFS-mandatory)</div>
<div> start vmfsFS then start nfs-server (kind:Mandatory) (id:order-vmfsFS-nfs-server-mandatory)</div>
<div> start ClusterIP then start nfs-server (kind:Mandatory) (id:order-ClusterIP-nfs-server-mandatory)</div>
<div>Colocation Constraints:</div>
<div> ms_drbd_vmfs with ClusterIP (score:INFINITY) (id:colocation-ms_drbd_vmfs-ClusterIP-INFINITY)</div>
<div> vmfsFS with ms_drbd_vmfs (score:INFINITY) (with-rsc-role:Master) (id:colocation-vmfsFS-ms_drbd_vmfs-INFINITY)</div>
<div> nfs-server with vmfsFS (score:INFINITY) (id:colocation-nfs-server-vmfsFS-INFINITY)</div>
<div><br>
</div>
<div>Cluster Properties:</div>
<div> cluster-infrastructure: corosync</div>
<div> cluster-name: fx201-vmcl</div>
<div> dc-version: 1.1.13-a14efad</div>
<div> have-watchdog: false</div>
<div> last-lrm-refresh: 1442528181</div>
<div> stonith-enabled: false</div>
</div>
<div><br>
</div>
<div>And status:</div>
<div><br>
</div>
<div>
<div>[root@fx201-1a log]# pcs status --full</div>
<div>Cluster name: fx201-vmcl</div>
<div>Last updated: Thu Sep 17 17:55:56 2015<span style="white-space:pre-wrap"> </span>
Last change: Thu Sep 17 17:18:10 2015 by root via crm_attribute on fx201-1b.ams</div>
<div>Stack: corosync</div>
<div>Current DC: fx201-1b.ams (2) (version 1.1.13-a14efad) - partition with quorum</div>
<div>2 nodes and 5 resources configured</div>
<div><br>
</div>
<div>Online: [ fx201-1a.ams (1) fx201-1b.ams (2) ]</div>
<div><br>
</div>
<div>Full list of resources:</div>
<div><br>
</div>
<div> ClusterIP<span style="white-space:pre-wrap"> </span>(ocf::heartbeat:IPaddr2):<span style="white-space:pre-wrap"></span>Started fx201-1a.ams</div>
<div> Master/Slave Set: ms_drbd_vmfs [drbd_vmfs]</div>
<div> drbd_vmfs<span style="white-space:pre-wrap"> </span>(ocf::linbit:drbd):<span style="white-space:pre-wrap"></span>Master fx201-1a.ams</div>
<div> drbd_vmfs<span style="white-space:pre-wrap"> </span>(ocf::linbit:drbd):<span style="white-space:pre-wrap"></span>Stopped</div>
<div> Masters: [ fx201-1a.ams ]</div>
<div> Stopped: [ fx201-1b.ams ]</div>
<div> vmfsFS<span style="white-space:pre-wrap"> </span>(ocf::heartbeat:Filesystem):<span style="white-space:pre-wrap"></span>Started fx201-1a.ams</div>
<div> nfs-server<span style="white-space:pre-wrap"> </span>(systemd:nfs-server):<span style="white-space:pre-wrap"></span>Started fx201-1a.ams</div>
<div><br>
</div>
<div>PCSD Status:</div>
<div> fx201-1a.ams: Online</div>
<div> fx201-1b.ams: Online</div>
<div><br>
</div>
<div>Daemon Status:</div>
<div> corosync: active/enabled</div>
<div> pacemaker: active/enabled</div>
<div> pcsd: active/enabled</div>
</div>
<div><br>
</div>
<div>If I do a failover, after manually confirming that the DRBD data is synchronized completely, it does work, but then never reconnects the secondary side, and in order to get the resource synchronized again, I have to manually correct it, ad infinitum.
I have tried standby/unstandby, pcs resource debug-start (with undesirable results), and so on. </div>
<div><br>
</div>
<div>Here are some relevant log messages from pacemaker.log:</div>
<div><br>
</div>
</div>
</div>
<div>
<div>
<div>
<div>Sep 17 17:48:10 [13954] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> crmd: info: crm_timer_popped:<span style="white-space:pre-wrap"></span>PEngine Recheck Timer (I_PE_CALC) just popped (900000ms)</div>
<div>Sep 17 17:48:10 [13954] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> crmd: notice: do_state_transition:<span style="white-space:pre-wrap"></span>State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]</div>
<div>Sep 17 17:48:10 [13954] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> crmd: info: do_state_transition:<span style="white-space:pre-wrap"></span>Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: process_pe_message:<span style="white-space:pre-wrap"></span>Input has not changed since last time, not saving to disk</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: determine_online_status:<span style="white-space:pre-wrap"></span>Node fx201-1b.ams is online</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: determine_online_status:<span style="white-space:pre-wrap"></span>Node fx201-1a.ams is online</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: determine_op_status:<span style="white-space:pre-wrap"></span>Operation monitor found resource drbd_vmfs:0 active in master mode on fx201-1b.ams</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: determine_op_status:<span style="white-space:pre-wrap"></span>Operation monitor found resource drbd_vmfs:0 active on fx201-1a.ams</div>
</div>
</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: native_print:<span style="white-space:pre-wrap"></span>ClusterIP<span style="white-space:pre-wrap"></span>(ocf::heartbeat:IPaddr2):<span style="white-space:pre-wrap"></span>Started fx201-1a.ams</div>
<span>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: clone_print:<span style="white-space:pre-wrap"></span>Master/Slave Set: ms_drbd_vmfs [drbd_vmfs]</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: short_print:<span style="white-space:pre-wrap"></span> Masters: [ fx201-1a.ams ]</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: short_print:<span style="white-space:pre-wrap"></span> Stopped: [ fx201-1b.ams ]</div>
</span>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: native_print:<span style="white-space:pre-wrap"></span>vmfsFS<span style="white-space:pre-wrap"></span>(ocf::heartbeat:Filesystem):<span style="white-space:pre-wrap"></span>Started fx201-1a.ams</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: native_print:<span style="white-space:pre-wrap"></span>nfs-server<span style="white-space:pre-wrap"></span>(systemd:nfs-server):<span style="white-space:pre-wrap"></span>Started fx201-1a.ams</div>
<span>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: native_color:<span style="white-space:pre-wrap"></span>Resource drbd_vmfs:1 cannot run anywhere</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: master_color:<span style="white-space:pre-wrap"></span>Promoting drbd_vmfs:0 (Master fx201-1a.ams)</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: master_color:<span style="white-space:pre-wrap"></span>ms_drbd_vmfs: Promoted 1 instances of a possible 1 to master</div>
</span>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: LogActions:<span style="white-space:pre-wrap"></span>Leave ClusterIP<span style="white-space:pre-wrap"></span>(Started fx201-1a.ams)</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: LogActions:<span style="white-space:pre-wrap"></span>Leave drbd_vmfs:0<span style="white-space:pre-wrap"></span>(Master fx201-1a.ams)</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: LogActions:<span style="white-space:pre-wrap"></span>Leave drbd_vmfs:1<span style="white-space:pre-wrap"></span>(Stopped)</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: LogActions:<span style="white-space:pre-wrap"></span>Leave vmfsFS<span style="white-space:pre-wrap"></span>(Started fx201-1a.ams)</div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: info: LogActions:<span style="white-space:pre-wrap"></span>Leave nfs-server<span style="white-space:pre-wrap"></span>(Started fx201-1a.ams)</div>
<div>
<div>
<div>Sep 17 17:48:10 [5662] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> pengine: notice: process_pe_message:<span style="white-space:pre-wrap"></span>Calculated Transition 16: /var/lib/pacemaker/pengine/pe-input-61.bz2</div>
<div>Sep 17 17:48:10 [13954] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> crmd: info: do_state_transition:<span style="white-space:pre-wrap"></span>State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]</div>
<div>Sep 17 17:48:10 [13954] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> crmd: info: do_te_invoke:<span style="white-space:pre-wrap"></span>Processing graph 16 (ref=pe_calc-dc-1442530090-97) derived from /var/lib/pacemaker/pengine/pe-input-61.bz2</div>
<div>Sep 17 17:48:10 [13954] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> crmd: notice: run_graph:<span style="white-space:pre-wrap"></span>Transition 16 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-61.bz2): Complete</div>
<div>Sep 17 17:48:10 [13954] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> crmd: info: do_log:<span style="white-space:pre-wrap"></span>FSA: Input I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE</div>
<div>Sep 17 17:48:10 [13954] <a href="http://fx201-1b.ams.accertify.net" target="_blank">
fx201-1b.ams.accertify.net</a> crmd: notice: do_state_transition:<span style="white-space:pre-wrap"></span>State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]</div>
</div>
</div>
</div>
<div>
<div>
<div><br>
</div>
<div>Thank you all for your help,</div>
<div><br>
</div>
<div>Jason</div>
</div>
</div>
</div>
<div>
<div>
<pre>"This message and any attachments may contain confidential information. If you
have received this message in error, any use or distribution is prohibited.
Please notify us by reply e-mail if you have mistakenly received this message,
and immediately and permanently delete it and any attachments. Thank you."</pre>
<br>
_______________________________________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank">Users@clusterlabs.org</a><br>
<a href="http://clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://clusterlabs.org/mailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">
http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">
http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</span>
<div>
<div>
<pre>"This message and any attachments may contain confidential information. If you
have received this message in error, any use or distribution is prohibited.
Please notify us by reply e-mail if you have mistakenly received this message,
and immediately and permanently delete it and any attachments. Thank you."</pre>
</div>
</div>
</div>
</div>
</span></div>
<div>
<div>
<pre>"This message and any attachments may contain confidential information. If you
have received this message in error, any use or distribution is prohibited.
Please notify us by reply e-mail if you have mistakenly received this message,
and immediately and permanently delete it and any attachments. Thank you."</pre>
</div>
</div>
<br>
_______________________________________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank">Users@clusterlabs.org</a><br>
<a href="http://clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://clusterlabs.org/mailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">
http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">
http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</span></div>
<div class="HOEnZb">
<div class="h5">
<pre>"This message and any attachments may contain confidential information. If you
have received this message in error, any use or distribution is prohibited.
Please notify us by reply e-mail if you have mistakenly received this message,
and immediately and permanently delete it and any attachments. Thank you."</pre>
</div>
</div>
<br>
_______________________________________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org">Users@clusterlabs.org</a><br>
<a href="http://clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://clusterlabs.org/mailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">
http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">
http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</span>
<pre>"This message and any attachments may contain confidential information. If you
have received this message in error, any use or distribution is prohibited.
Please notify us by reply e-mail if you have mistakenly received this message,
and immediately and permanently delete it and any attachments. Thank you."</pre>
</div>
</div>
</span>
<pre>"This message and any attachments may contain confidential information. If you
have received this message in error, any use or distribution is prohibited.
Please notify us by reply e-mail if you have mistakenly received this message,
and immediately and permanently delete it and any attachments. Thank you."</pre>
</div>
</div>
</span>
</body>
</html>
<pre>
"This message and any attachments may contain confidential information. If you
have received this message in error, any use or distribution is prohibited.
Please notify us by reply e-mail if you have mistakenly received this message,
and immediately and permanently delete it and any attachments. Thank you."</pre>