<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
code
{mso-style-priority:99;
font-family:"Courier New";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";
mso-fareast-language:EN-GB;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Dear all,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I’m pretty new to Pacemaker so I must be missing something but I cannot find it in the documentation.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I’m setting up a SAMBA File Server cluster with DRBD and Pacemaker. Here are the relevant pcs commands related to the mount part:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:9.5pt;font-family:"Courier New";color:#4E9A06;border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">user $
</span></b><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">sudo pcs cluster cib fs_cfg</span><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif;color:white;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:9.5pt;font-family:"Courier New";color:#4E9A06;border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">user $
</span></b><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">sudo pcs -f fs_cfg resource create VPSFSMount Filesystem device="/dev/drbd1" directory="/srv/vps-fs" fstype="gfs2" "options=acl,noatime"</span><span style="font-size:10.5pt;font-family:"Helvetica",sans-serif;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal" style="word-break:break-all"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"> Assumed agent name 'ocf:heartbeat:Filesystem' (deduced from 'Filesystem')<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">It all works fine, here is an extract of the pcs status command:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b><span style="font-size:9.5pt;font-family:"Courier New";color:#4E9A06;border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">user $</span></b>
<span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">
sudo pcs status<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Cluster name: vps-fs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Stack: corosync<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Current DC: vps-fs-04 (version 1.1.18-2b07d5c5a9) - partition with quorum<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Last updated: Tue Jul 14 11:13:55 2020<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Last change: Tue Jul 14 10:31:36 2020 by root via cibadmin on vps-fs-04<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">2 nodes configured<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">7 resources configured<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Online: [ vps-fs-03 vps-fs-04 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Full list of resources:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">stonith_vps-fs (stonith:external/ssh): Started vps-fs-04<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Clone Set: dlm-clone [dlm]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> Started: [ vps-fs-03 vps-fs-04 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Master/Slave Set: VPSFSClone [VPSFS]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> Masters: [ vps-fs-03 vps-fs-04 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Clone Set: VPSFSMount-clone [VPSFSMount]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> Started: [ vps-fs-03 vps-fs-04 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Daemon Status:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> corosync: active/enabled<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> pacemaker: active/enabled<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> pcsd: active/enabled<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal">I can start CTDB (SAMBA cluster manager manually) and it’s fine. However, CTDB shares a lock file between both nodes which is located on the shared mount point.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The problem comes from the moment I reboot one of the servers (vps-fs-04) and Pacemaker (and Corosync) are started automatically upon boot (I’m talking about unexpected reboot, not maintenance reboot which I didn’t try yet).<o:p></o:p></p>
<p class="MsoNormal">After reboot, the server (vps-fs-04) comes back online and in the cluster but the one that wasn’t rebooted has an issue with the mount resource:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b><span style="font-size:9.5pt;font-family:"Courier New";color:#4E9A06;border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">user $
</span></b><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">sudo pcs status<b><span style="color:#4E9A06"><o:p></o:p></span></b></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Cluster name: vps-fs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Stack: corosync<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Current DC: vps-fs-03 (version 1.1.18-2b07d5c5a9) - partition with quorum<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Last updated: Tue Jul 14 11:33:44 2020<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Last change: Tue Jul 14 10:31:36 2020 by root via cibadmin on vps-fs-04<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">2 nodes configured<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">7 resources configured<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Node vps-fs-03: UNCLEAN (online)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Online: [ vps-fs-04 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Full list of resources:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">stonith_vps-fs (stonith:external/ssh): Started vps-fs-03<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Clone Set: dlm-clone [dlm]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> Started: [ vps-fs-03 vps-fs-04 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Master/Slave Set: VPSFSClone [VPSFS]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> Masters: [ vps-fs-03 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> Slaves: [ vps-fs-04 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Clone Set: VPSFSMount-clone [VPSFSMount]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> VPSFSMount (ocf::heartbeat:Filesystem): FAILED vps-fs-03<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> Stopped: [ vps-fs-04 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Failed Actions:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">* VPSFSMount_stop_0 on vps-fs-03 'unknown error' (1): call=65, status=Timed Out, exitreason='Couldn't unmount /srv/vps-fs;
trying cleanup with KILL',<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> last-rc-change='Tue Jul 14 11:23:46 2020', queued=0ms, exec=60011ms<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">Daemon Status:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> corosync: active/enabled<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> pacemaker: active/enabled<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"> pcsd: active/enabled<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal">The problem seems to come from the fact that the mount point (/srv/vps-fs) is busy (probably CTDB lock file) but what I don’t understand is why does the server not rebooted (vps-fs-03) need to remount an already mounted file system when
the other node comes back online.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’ve checked the <span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB">
'ocf:heartbeat:Filesystem' </span>documentation but nothing seemed to help. The only thing I did was to change the following:<span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New";mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:9.5pt;font-family:"Courier New";color:#4E9A06;border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">user $
</span></b><span style="font-size:9.5pt;font-family:"Courier New";border:none windowtext 1.0pt;padding:0cm;mso-fareast-language:EN-GB">sudo pcs resource update VPSFSMount fast_stop="no" op monitor timeout="60"<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">However this didn’t help. Google doesn’t give me much help either (but maybe I’m not searching for the right thing).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thank you in advance for any pointer!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Kr,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Gregory<o:p></o:p></p>
</div>
</body>
</html>