<div dir="ltr"><div>I recently had some time to read the documentation for LVM Shared Storage using PCS and found that there wasn't any documentation on how to set it up using lvmlockd/GFS2/Fencing in a shared configuration. There are a couple replication scenarios out there that use DRDB/Cloning which I might try if this isn't possible.<br></div><div><br></div><div>Everything I have found is CLVMD based or is incredibly old created way before LVMLockD deprecated CLVMD. So I decided to take the guessing route:</div><div><br></div><div>So this is how far I got:<br></div><div><br></div><div>1) I created a LVM volume with GFS2 physically formatted volumes.</div><div><span style="font-family:monospace"><span style="font-weight:bold;color:rgb(255,84,84);background-color:rgb(255,255,255)">vgcreate</span><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"> "vmgfs2" /dev/sdb1 /dev/sdc1 /dev/sde1 /dev/sdg1 --shared</span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="font-weight:bold;color:rgb(255,84,84);background-color:rgb(255,255,255)">lvcreate</span><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"> -n "smvms" -L 50G "vmgfs2"</span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">2) I formatted a GFS2 file system on the smvs segment:</span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="font-weight:bold;color:rgb(255,84,84);background-color:rgb(255,255,255)">mkfs.gfs2</span><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"> -p lock_dlm -t vmgfs2:smvms -j 8 /dev/vmgfs2/smvms</span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">3) I then fenced the block devices with stonith like so:</span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">
<span></span>
</span></span></span></span></span></span><p style="line-height:100%;margin-bottom:0in;background:transparent">
pcs stonith create scsi-shooter fence_scsi
pcmk_host_list="hyperv0001.cluster hyperv0002.cluster"
devices=/dev/disk/by-id/wwn-0x6001405f0efeeeb51af42c09971344de,/dev/disk/by-id/wwn-0x6001405fbe61a5c9e234a81968805ac3,/dev/disk/by-id/wwn-0x60014055c9a285a61214064a7ad60ccd,/dev/disk/by-id/wwn-0x600140565c8b9cb3f874d7f89ef4afa1
meta provides=unfencing</p>
</div><div><br></div><div>4) Now for the lvm.conf configuration, that part wasn't obvious. Since I do not have any lvm volumes on the hypervisors other than the block devices assigned specifically for shared storage I didn't have any filters setup. (i.e. the hypervisors main block devices are nvme and the other block devices are not part of a shared volume.)</div><div><br></div><div>That being said I set the following up in lvm.conf:</div><div><span style="font-family:monospace"><span style="color:rgb(175,95,0);background-color:rgb(255,255,255)">use_lvmlockd</span><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"> = 1</span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(175,95,0);background-color:rgb(255,255,255)">system_id_source</span><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"> = </span><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)">"none"</span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(175,95,0);background-color:rgb(255,255,255)">volume_list</span><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"> = []</span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">Some more info:</span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># vgs
</span><br> VG #PV #LV #SN Attr VSize VFree <br> vmgfs2 4 1 0 wz--ns <1023.91g <973.91g<br></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace">
and</span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># vgs
</span><br> Reading VG vmgfs2 without a lock.
<br> VG #PV #LV #SN Attr VSize VFree <br> vmgfs2 4 1 0 wz--ns <1023.91g <973.91g<br>
<br></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="font-family:monospace">I assume they both have to have a shared attribute set.<br></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><br></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace">Both machines boot with lvmlockd enabled at boot time.</span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><br></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace">It also was suggested I setup my initrd on the hypervisors to load the lvm.conf file at startup with the proper configuration options more immediate.</span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><br></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace">So I then updated my initrd like so:</span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-weight:bold;color:rgb(255,84,84);background-color:rgb(255,255,255)">dracut</span><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"> -H -f /boot/initramfs-$(uname -r).img $(uname -r)</span><br></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">5) Next I defined a activation resource, which I took from some relatively recent documentation I found out on the web:</span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">pcs </span><span style="font-weight:bold;color:rgb(255,84,84);background-color:rgb(255,255,255)">resource</span><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"> create vmgfs2_pcs ocf:heartbeat:LVM-activate vgname=vmgfs2 vg_access_mode=lvmlockd --group vmgfs2</span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">This however leads to some issues:</span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">Cluster name: vmgfs2
</span><br>Cluster Summary:
<br> * Stack: corosync
<br> * Current DC: <a href="http://hyperv0001.aesgi.com">hyperv0001.aesgi.com</a> (version 2.1.4-4.fc36-dc6eb4362e) - partition with quorum
<br> * Last updated: Fri Dec 30 17:26:45 2022
<br> * Last change: Fri Dec 30 14:48:38 2022 by root via cibadmin on <a href="http://hyperv0001.aesgi.com">hyperv0001.aesgi.com</a>
<br> * 2 nodes configured
<br> * 4 resource instances configured
<br>
<br>Node List:
<br> * Online: [ <a href="http://hyperv0001.aesgi.com">hyperv0001.aesgi.com</a> <a href="http://hyperv0002.aesgi.com">hyperv0002.aesgi.com</a> ]
<br>
<br>Full List of Resources:
<br> * Clone Set: dlm-clone [dlm]:
<br> * Started: [ <a href="http://hyperv0001.aesgi.com">hyperv0001.aesgi.com</a> <a href="http://hyperv0002.aesgi.com">hyperv0002.aesgi.com</a> ]
<br> * scsi-shooter (stonith:fence_scsi): Started <a href="http://hyperv0001.aesgi.com">hyperv0001.aesgi.com</a>
<br> * Resource Group: vmgfs2:
<br> * vmgfs2_pcs (ocf::heartbeat:LVM-activate): Stopped
<br>
<br>Failed Resource Actions:
<br> * vmgfs2_pcs_start_0 on <a href="http://hyperv0002.aesgi.com">hyperv0002.aesgi.com</a> 'not configured' (6): call=16, status='complete', exitreason='lvmlockd daemon is not running!', last-rc-chan<br>ge='Fri Dec 30 14:53:17 2022', queued=0ms, exec=1429ms
<br>
<br>Daemon Status:
<br> corosync: active/enabled
<br> pacemaker: active/enabled
<br> pcsd: active/enabled<br>
<br></span></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace">I am not sure why I get a lvmlockd not running response as it is configured to run at bootup.</span></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><br></span></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace">But since I used a variety of locations to put together this information, I was wondering if there wasn't a location that already has this specified?</span></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><br></span></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><br></span></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><br></span></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"> <br></span></span></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(135,0,95);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></span></span></span></span></div><div><br></div><div><br></div></div>