<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"ＭＳ 明朝";}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:Calibri;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:Calibri;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">So, I’m setting up a two node cluster that will eventually (hopefully) serve as an HA iSCSI Target (Active/Passive) on RHEL 6. I’m using the [incredibly poorly written] guide I found on Linbit’s website (“Highly
available iSCSI storage with DRBD and Pacemaker”). I have somehow gotten pretty far through it, but I’ve hit a couple of snags.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Here’s my drbd.conf (/etc/drbd.d/global_common.conf):<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">global {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> usage-count yes;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # minor-count dialog-refresh disable-ip-verification<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">common {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> protocol C;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> handlers {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # These are EXAMPLE handlers only.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # They may have severe implications,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # like hard resetting the node under certain circumstances.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # Be careful when chosing your poison.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # fence-peer "/usr/lib/drbd/crm-fence-peer.sh";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # split-brain "/usr/lib/drbd/notify-split-brain.sh root";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # before-resync-target "/usr/lib/drbd/snapshot-resync-target-lvm.sh -p 15 -- -c 16k";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> startup {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # wfc-timeout degr-wfc-timeout outdated-wfc-timeout wait-after-sb<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> disk {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # on-io-error fencing use-bmbv no-disk-barrier no-disk-flushes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # no-disk-drain no-md-flushes max-bio-bvecs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> net {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # sndbuf-size rcvbuf-size timeout connect-int ping-int ping-timeout max-buffers<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # max-epoch-size ko-count allow-two-primaries cram-hmac-alg shared-secret<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # after-sb-0pri after-sb-1pri after-sb-2pri data-integrity-alg no-tcp-cork<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> syncer {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # rate after al-extents use-rle cpu-mask verify-alg csums-alg<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">And here’s my targetfs.res config (/etc/drbd.d/targetfs.res):<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">resource targetfs {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">protocol C;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">meta-disk internal;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">device /dev/drbd1;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">disk /dev/xvdf;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">syncer {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> verify-alg sha1;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> c-plan-ahead 0;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> rate 32M;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">net {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> allow-two-primaries;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">on node1 {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> address 10.130.96.120:7789;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">on node2 {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> address 10.130.97.165:7789;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">These, of course, live on both nodes.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Once I create the drbd md and sync the nodes:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">(node1)# drbdadm create-md targetfs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">(node2)# drbdadm create-md targetfs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">(node1)# drbdadm up targetfs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">(node2)# drbdadm up targetfs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">(node2)# drbdadm invalidate targetfs
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">(node1)# cat /proc/drbd<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">version: 8.3.16 (api:88/proto:86-97)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6, 2014-11-24 14:51:37<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> ns:134213632 nr:0 dw:36 dr:134215040 al:1 bm:8192 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I run a pvcreate and an lvcreate on node1:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">(node1)# pvcreate /dev/drbd/by-res/targetfs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">(node1)# vgcreate targetfs /dev/drbd/by-res/targetfs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">(node1)# pvs && vgs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> PV VG Fmt Attr PSize PFree
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> /dev/drbd1 targetfs lvm2 a--u 127.99g 127.99g<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> VG #PV #LV #SN Attr VSize VFree
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> targetfs 1 0 0 wz--n- 127.99g 127.99g<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">pcs cluster configuration goes well enough for a bit:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs cluster setup --name gctvanas node1 node2 --transport udpu<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs cluster start --all<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs property set stonith-enabled=false<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs property set no-quorum-policy=ignore<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs property set default-resource-stickiness="200"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs resource create gctvanas-vip ocf:heartbeat:IPaddr2 ip=10.30.96.100 cidr_netmask=32 nic=eth0 op monitor interval=30s<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs cluster cib drbd_cfg<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs -f drbd_cfg resource create gctvanas-fs1o ocf:linbit:drbd drbd_resource=targetfs op monitor interval=10s<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs -f drbd_cfg resource master gctvanas-fs2o gctvanas-fs1o master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs cluster cib-push drbd_cfg<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs status<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Cluster name: gctvanas<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Stack: cman<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Current DC: node1 (version 1.1.15-1.9a34920.git.el6-9a34920) - partition with quorum<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Last updated: Fri Aug 26 11:29:11 2016<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Last change: Fri Aug 26 11:29:07 2016 by root via cibadmin on node1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">2 nodes configured<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">3 resources configured<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Online: [ node1 node2 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Full list of resources:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Master/Slave Set: gctvanas-fs2o [gctvanas-fs1o]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> Masters: [ node1 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> Slaves: [ node2 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">gctvanas-vip (ocf::heartbeat:IPaddr2): Started node1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">PCSD Status:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> node1: Online<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> node2: Online<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">And then I do this:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs resource create gctvanas-lvm ocf:heartbeat:LVM params volgrpname=targetfs op monitor interval="30s”<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">and the wheels come off the cart :-|<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># pcs status<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Cluster name: gctvanas<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Stack: cman<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Current DC: node1 (version 1.1.15-1.9a34920.git.el6-9a34920) - partition with quorum<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Last updated: Fri Aug 26 11:27:29 2016<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Last change: Fri Aug 26 10:57:21 2016 by root via cibadmin on node1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">2 nodes configured<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">4 resources configured<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Online: [ node1 node2 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Full list of resources:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Master/Slave Set: gctvanas-fs2o [gctvanas-fs1o]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> Masters: [ node1 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> Slaves: [ node2 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">gctvanas-vip (ocf::heartbeat:IPaddr2): Started node1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">gctvanas-lvm (ocf::heartbeat:LVM): Stopped<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Failed Actions:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">* gctvanas-lvm_start_0 on node1 'not running' (7): call=42, status=complete, exitreason='LVM: targetfs did not activate correctly',<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> last-rc-change='Fri Aug 26 10:57:22 2016', queued=0ms, exec=577ms<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">* gctvanas-lvm_start_0 on node2 'unknown error' (1): call=34, status=complete, exitreason='Volume group [targetfs] does not exist or contains error! Volume group "targetfs" not found',<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> last-rc-change='Fri Aug 26 10:57:21 2016', queued=0ms, exec=322ms<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">PCSD Status:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> node1: Online<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> node2: Online<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I’m not seeing anything obvious on node1 that indicates what the issue is there, but the error on node2 makes a little more sense since node2 doesn’t actually know about the pv/vg created on the drbd disk
(so I feel like I’m missing a step that somehow lets node2 know what’s going on there). Any ideas?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Here’s my current cluster.conf<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><cluster config_version="9" name="gctvanas"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <fence_daemon/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <clusternodes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <clusternode name="node1" nodeid="1"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <fence><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <method name="pcmk-method"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <device name="pcmk-redirect" port="node1"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </method><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </fence><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </clusternode><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <clusternode name="node2" nodeid="2"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <fence><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <method name="pcmk-method"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <device name="pcmk-redirect" port="node2"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </method><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </fence><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </clusternode><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </clusternodes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <cman broadcast="no" expected_votes="1" transport="udpu" two_node="1"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <fencedevices><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <fencedevice agent="fence_pcmk" name="pcmk-redirect"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </fencedevices><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <rm><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <failoverdomains/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <resources/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </rm><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"></cluster><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Here’s my current cib dump:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><cib crm_feature_set="3.0.11" validate-with="pacemaker-2.6" epoch="56" num_updates="10" admin_epoch="0" cib-last-written="Fri Aug 26 10:57:21 2016" update-origin="node1" update-client="cibadmin" update-user="root"
have-quorum="1" dc-uuid="node1"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <configuration><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <crm_config><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <cluster_property_set id="cib-bootstrap-options"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="cib-bootstrap-options-have-watchdog" name="have-watchdog" value="false"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.15-1.9a34920.git.el6-9a34920"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="cman"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" value="ignore"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="cib-bootstrap-options-default-resource-stickiness" name="default-resource-stickiness" value="200"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="cib-bootstrap-options-last-lrm-refresh" name="last-lrm-refresh" value="1472203780"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </cluster_property_set><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </crm_config><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nodes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <node id="node2" uname="node2"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <node id="node1" uname="node1"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </nodes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <resources><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <master id="gctvanas-fs2o"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <primitive class="ocf" id="gctvanas-fs1o" provider="linbit" type="drbd"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <instance_attributes id="gctvanas-fs1o-instance_attributes"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="gctvanas-fs1o-instance_attributes-drbd_resource" name="drbd_resource" value="targetfs"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </instance_attributes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <operations><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <op id="gctvanas-fs1o-start-interval-0s" interval="0s" name="start" timeout="240"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <op id="gctvanas-fs1o-promote-interval-0s" interval="0s" name="promote" timeout="90"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <op id="gctvanas-fs1o-demote-interval-0s" interval="0s" name="demote" timeout="90"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <op id="gctvanas-fs1o-stop-interval-0s" interval="0s" name="stop" timeout="100"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <op id="gctvanas-fs1o-monitor-interval-10s" interval="10s" name="monitor"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </operations><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </primitive><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <meta_attributes id="gctvanas-fs2o-meta_attributes"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="gctvanas-fs2o-meta_attributes-master-max" name="master-max" value="1"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="gctvanas-fs2o-meta_attributes-master-node-max" name="master-node-max" value="1"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="gctvanas-fs2o-meta_attributes-clone-max" name="clone-max" value="2"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="gctvanas-fs2o-meta_attributes-clone-node-max" name="clone-node-max" value="1"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="gctvanas-fs2o-meta_attributes-notify" name="notify" value="true"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </meta_attributes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </master><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <primitive class="ocf" id="gctvanas-vip" provider="heartbeat" type="IPaddr2"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <instance_attributes id="gctvanas-vip-instance_attributes"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="gctvanas-vip-instance_attributes-ip" name="ip" value="10.30.96.100"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="gctvanas-vip-instance_attributes-cidr_netmask" name="cidr_netmask" value="32"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="gctvanas-vip-instance_attributes-nic" name="nic" value="eth0"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </instance_attributes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <operations><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <op id="gctvanas-vip-start-interval-0s" interval="0s" name="start" timeout="20s"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <op id="gctvanas-vip-stop-interval-0s" interval="0s" name="stop" timeout="20s"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <op id="gctvanas-vip-monitor-interval-30s" interval="30s" name="monitor"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </operations><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </primitive><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <primitive class="ocf" id="gctvanas-lvm" provider="heartbeat" type="LVM"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <instance_attributes id="gctvanas-lvm-instance_attributes"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="gctvanas-lvm-instance_attributes-volgrpname" name="volgrpname" value="targetfs"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </instance_attributes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <operations><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <op id="gctvanas-lvm-start-interval-0s" interval="0s" name="start" timeout="30"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <op id="gctvanas-lvm-stop-interval-0s" interval="0s" name="stop" timeout="30"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <op id="gctvanas-lvm-monitor-interval-30s" interval="30s" name="monitor"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </operations><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </primitive><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </resources><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <constraints/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <rsc_defaults><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <meta_attributes id="rsc_defaults-options"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="rsc_defaults-options-resource-stickiness" name="resource-stickiness" value="100"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </meta_attributes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </rsc_defaults><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </configuration><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <status><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <node_state id="node1" uname="node1" in_ccm="true" crmd="online" crm-debug-origin="do_update_resource" join="member" expected="member"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm id="node1"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_resources><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_resource id="gctvanas-vip" type="IPaddr2" class="ocf" provider="heartbeat"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-vip_last_0" operation_key="gctvanas-vip_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="32:1:0:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:0;32:1:0:681b3ca7-f83d-4396-a249-d6d80e0efe16" on_node="node1" call-id="11" rc-code="0" op-status="0" interval="0" last-run="1472203004" last-rc-change="1472203004" exec-time="74" queue-time="0" op-digest="15b4ba230497d33ad5d77f05e4b9a83e"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-vip_monitor_30000" operation_key="gctvanas-vip_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="33:1:0:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:0;33:1:0:681b3ca7-f83d-4396-a249-d6d80e0efe16" on_node="node1" call-id="14" rc-code="0" op-status="0" interval="30000" last-rc-change="1472203004" exec-time="72" queue-time="0" op-digest="205179ac48c643694ee24512cc3b1429"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </lrm_resource><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_resource id="gctvanas-fs1o" type="drbd" class="ocf" provider="linbit"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-fs1o_last_failure_0" operation_key="gctvanas-fs1o_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="3:9:7:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:8;3:9:7:681b3ca7-f83d-4396-a249-d6d80e0efe16" on_node="node1" call-id="37" rc-code="8" op-status="0" interval="0" last-run="1472203781" last-rc-change="1472203781" exec-time="37" queue-time="1" op-digest="fb1e24e691d75f64117224686c0f806b"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-fs1o_last_0" operation_key="gctvanas-fs1o_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="3:9:7:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:8;3:9:7:681b3ca7-f83d-4396-a249-d6d80e0efe16" on_node="node1" call-id="37" rc-code="8" op-status="0" interval="0" last-run="1472203781" last-rc-change="1472203781" exec-time="37" queue-time="1" op-digest="fb1e24e691d75f64117224686c0f806b"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </lrm_resource><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_resource id="gctvanas-lvm" type="LVM" class="ocf" provider="heartbeat"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-lvm_last_0" operation_key="gctvanas-lvm_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="2:36:0:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:0;2:36:0:681b3ca7-f83d-4396-a249-d6d80e0efe16" on_node="node1" call-id="43" rc-code="0" op-status="0" interval="0" last-run="1472223443" last-rc-change="1472223443" exec-time="369" queue-time="0" op-digest="df48f11b20123e34ddf99999ce9f3f1c"
exit-reason="LVM: targetfs did not activate correctly"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-lvm_last_failure_0" operation_key="gctvanas-lvm_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="37:35:0:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:7;37:35:0:681b3ca7-f83d-4396-a249-d6d80e0efe16" exit-reason="LVM: targetfs did not activate correctly" on_node="node1" call-id="42" rc-code="7" op-status="0" interval="0" last-run="1472223442" last-rc-change="1472223442" exec-time="577"
queue-time="0" op-digest="df48f11b20123e34ddf99999ce9f3f1c"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </lrm_resource><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </lrm_resources><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </lrm><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <transient_attributes id="node1"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <instance_attributes id="status-node1"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="status-node1-shutdown" name="shutdown" value="0"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="status-node1-last-failure-gctvanas-fs1o" name="last-failure-gctvanas-fs1o" value="1472203145"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="status-node1-master-gctvanas-fs1o" name="master-gctvanas-fs1o" value="10000"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="status-node1-fail-count-gctvanas-lvm" name="fail-count-gctvanas-lvm" value="INFINITY"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="status-node1-last-failure-gctvanas-lvm" name="last-failure-gctvanas-lvm" value="1472223442"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </instance_attributes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </transient_attributes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </node_state><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <node_state id="node2" uname="node2" in_ccm="true" crmd="online" crm-debug-origin="do_update_resource" join="member" expected="member"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm id="node2"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_resources><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_resource id="gctvanas-vip" type="IPaddr2" class="ocf" provider="heartbeat"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-vip_last_0" operation_key="gctvanas-vip_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="5:0:7:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:7;5:0:7:681b3ca7-f83d-4396-a249-d6d80e0efe16" on_node="node2" call-id="10" rc-code="7" op-status="0" interval="0" last-run="1472203004" last-rc-change="1472203004" exec-time="63" queue-time="0" op-digest="15b4ba230497d33ad5d77f05e4b9a83e"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </lrm_resource><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_resource id="gctvanas-fs1o" type="drbd" class="ocf" provider="linbit"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-fs1o_last_failure_0" operation_key="gctvanas-fs1o_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="4:9:7:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:0;4:9:7:681b3ca7-f83d-4396-a249-d6d80e0efe16" on_node="node2" call-id="28" rc-code="0" op-status="0" interval="0" last-run="1472203781" last-rc-change="1472203781" exec-time="39" queue-time="0" op-digest="fb1e24e691d75f64117224686c0f806b"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-fs1o_last_0" operation_key="gctvanas-fs1o_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="4:9:7:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:0;4:9:7:681b3ca7-f83d-4396-a249-d6d80e0efe16" on_node="node2" call-id="28" rc-code="0" op-status="0" interval="0" last-run="1472203781" last-rc-change="1472203781" exec-time="39" queue-time="0" op-digest="fb1e24e691d75f64117224686c0f806b"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-fs1o_monitor_10000" operation_key="gctvanas-fs1o_monitor_10000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="9:10:0:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:0;9:10:0:681b3ca7-f83d-4396-a249-d6d80e0efe16" on_node="node2" call-id="29" rc-code="0" op-status="0" interval="10000" last-rc-change="1472203781" exec-time="45" queue-time="0" op-digest="1a2c711d2933e74557b4c1a13ec62162"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </lrm_resource><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_resource id="gctvanas-lvm" type="LVM" class="ocf" provider="heartbeat"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-lvm_last_0" operation_key="gctvanas-lvm_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="3:35:0:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:0;3:35:0:681b3ca7-f83d-4396-a249-d6d80e0efe16" on_node="node2" call-id="35" rc-code="0" op-status="0" interval="0" last-run="1472223442" last-rc-change="1472223442" exec-time="151" queue-time="0" op-digest="df48f11b20123e34ddf99999ce9f3f1c"
exit-reason="Volume group [targetfs] does not exist or contains error! Volume group "targetfs" not found"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <lrm_rsc_op id="gctvanas-lvm_last_failure_0" operation_key="gctvanas-lvm_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="38:33:0:681b3ca7-f83d-4396-a249-d6d80e0efe16"
transition-magic="0:1;38:33:0:681b3ca7-f83d-4396-a249-d6d80e0efe16" exit-reason="Volume group [targetfs] does not exist or contains error! Volume group "targetfs" not found" on_node="node2" call-id="34" rc-code="1" op-status="0" interval="0" last-run="1472223441"
last-rc-change="1472223441" exec-time="322" queue-time="0" op-digest="df48f11b20123e34ddf99999ce9f3f1c"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </lrm_resource><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </lrm_resources><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </lrm><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <transient_attributes id="node2"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <instance_attributes id="status-node2"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="status-node2-shutdown" name="shutdown" value="0"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="status-node2-last-failure-gctvanas-fs1o" name="last-failure-gctvanas-fs1o" value="1472203144"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="status-node2-master-gctvanas-fs1o" name="master-gctvanas-fs1o" value="10000"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="status-node2-fail-count-gctvanas-lvm" name="fail-count-gctvanas-lvm" value="INFINITY"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <nvpair id="status-node2-last-failure-gctvanas-lvm" name="last-failure-gctvanas-lvm" value="1472223441"/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </instance_attributes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </transient_attributes><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </node_state><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </status><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"></cib><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F4E79">--</span><span style="font-size:11.0pt;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F4E79"> </span><span style="font-size:11.0pt;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:14.0pt;color:#27628C">[ jR ]</span></b><span style="font-size:14.0pt;color:#27628C"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F4E79"> @: <a href="mailto:jason@eramsey.org"><span style="color:blue">jason@eramsey.org</span></a></span><span style="font-size:11.0pt;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F4E79"> </span><span style="font-size:11.0pt;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F4E79"> </span><i><span style="font-size:9.0pt;color:#1F4E79">there is no path to greatness; greatness is the path</span></i><o:p></o:p></p>
</div>
</body>
</html>