<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:68961148;
mso-list-type:hybrid;
mso-list-template-ids:1063293304 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1
{mso-list-id:2123839484;
mso-list-type:hybrid;
mso-list-template-ids:-51993306 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Team,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This is an old issue that persists over different HA platforms:<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">Automounter (or autofs service) provides the ability of one FS to be mounted additionally on demand on specific location. Typical example of this is the home environment for different
applications<o:p></o:p></li><ul style="margin-top:0in" type="circle">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level2 lfo1">And app ID in naming services (LDAP, AD etc.) has one home environment – lets say “/home/appid”<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level2 lfo1">For different hosts this home environment might be different – on one host it can be “/export/home/appid/v1.0”, on another host it might be “/export/home/appid/v2.0” etc.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level2 lfo1">Still when the app is to execute something on a host it needs its home environment as listed in the naming layer, so we configure automounter to mount as /home/appid “/export/home/appid/v1.0”
for the first server and “/export/home/appid/v2.0” for the second server.<o:p></o:p></li></ul>
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">A directory used by automounter CANNOT BE USED by anyone else – any attempt to mount something under /home in the above example will show “device busy” message and will fail.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">File system mounted and then loopback mounted under autofs control looks like this:<o:p></o:p></li></ul>
<p class="MsoNormal" style="margin-left:1.0in">tlsys-ucs-eng08a:/appl/test # <b>cat /etc/auto.master</b><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># Sample auto.master file<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># Format of this file:<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># mountpoint map options<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># Also see variable AUTOFS_OPTIONS in /etc/sysconfig/autofs<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># For details of the format look at autofs(8).<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"><b><span style="color:red">/appl /etc/auto_appl -rw,intr,nosuid,nobrowse<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin-left:1.0in">…<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">tlsys-ucs-eng08a:/appl/test # <b>cat /etc/auto_appl</b><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># Local auto_appl automounter file.<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># Example:<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># directory --bind localhost:/export/appl/directory<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">#<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># When adding entries for NFS mounts from Solaris servers add the following<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># options:<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># -rsize=32768,wsize=32768,nfsvers=3,tcp,retrans=5,timeo=600<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># Example<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"># dir -rsize=32768,wsize=32768,nfsvers=3,tcp,retrans=5,timeo=600 server:/mount<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">…<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"><b><span style="color:red">test --bind localhost:/export/appl/test<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin-left:1.0in">tlsys-ucs-eng08a:/usr/lib/ocf/resource.d/heartbeat #
<b>cd /appl/test</b><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">tlsys-ucs-eng08a:/appl/test # <b>mount |grep test<o:p></o:p></b></p>
<p class="MsoNormal" style="margin-left:1.0in">/dev/mapper/DG1-test on /export/appl/test type ext4 (rw,relatime)<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"><b><span style="color:red">/dev/mapper/DG1-test on /appl/test type ext4 (rw,relatime</span></b>)<o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l1 level1 lfo2">If we try to unmount /export/appl/test in the above example we will get “device busy” message, but there will be no process in the processes table showing usage. Neither lsof will
show anything regarding this FS.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l1 level1 lfo2">In case of SLES HA, attempt to stop the resource or to migrate it to another server will cause panic to the server as the Filesystem agent will be unable to stop the resource.<o:p></o:p></li></ul>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The above behavior is not acceptable. We have configure multiple service groups that can be executed independently on any of the members of a HA cluster, so on one host we may have more than one services. Panic on the host would disrupt
the work of other applications.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">To avoid this I modified lightly the Filesystem agent, allowing it to search for such cases. As a base I will use version resource-agents-4.4.0+git57.70549516-3.12.1.x86_64:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">…<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">320 # Lists all filesystems potentially mounted under a given path,<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">321 # excluding the path itself.<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">322 list_submounts() {<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">323 list_mounts | grep " $1/" | cut -d' ' -f2 | sort -r<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">324 }<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">325<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"><b><span style="color:red">326 # FNMA - Lists automounter loopback<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin-left:1.0in"><b><span style="color:red">327 list_loopbacks() {<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin-left:1.0in"><b><span style="color:red">328 list_mounts | grep "$1" | grep -v "$2" | cut -d' ' -f2 | sort -r<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin-left:1.0in"><b><span style="color:red">329 }<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin-left:1.0in">330<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in">…<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:1.0in"><b><span style="color:red">649 # for SUB in `list_submounts $MOUNTPOINT` $MOUNTPOINT; do<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin-left:1.0in"><b><span style="color:red">650 # FNMA: original line above was modified bellow:<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin-left:1.0in"><b><span style="color:red">651 for SUB in `list_submounts $MOUNTPOINT` `list_loopbacks $DEVICE $MOUNTPOINT` $MOUNTPOINT; do<o:p></o:p></span></b></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Adding one extra subroutine to look for loopbacks, and during the buildup of the list of submounts adding it to address the issue. This makes graceful stop of the FS resource without the panic.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The logic of the proposed change is this – if we stop FS, we need to stop ANY its representations on the current host. Most likely this will be in preparation for the next step – to disable the vg for migration. Failure to release the device
that has the FS will prevent the vg to be disabled.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Attached is a copy that we at Fannie Mae use with no problems. The inconvenience for us is with each and every patch upgrade or major version release we need to redo the agent modification. I believe this small change deserves to be part
of the original code. Any thoughts?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Vladimir Yanakiev<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Unix Engineer, Hosting & Engineering Services – Solution Engineering Compute<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Phone: 703-833-3770 (direct) | 571-246-1946 (mobile)<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>