<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;
font-weight:normal;
font-style:normal;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">We have User jobs running at the time the split brain scenario occurs. The CPU load at that time is around 55 (We have 32 CPU cores). Is there any way we can avoid the split brain scenario in this case.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">With Regards<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Somanath Thilak J<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b>From:</b> Emmanuel Gelati <emi2fast@gmail.com> <br>
<b>Sent:</b> Monday, June 24, 2019 01:57<br>
<b>To:</b> Cluster Labs - All topics related to open-source clustering welcomed <users@clusterlabs.org><br>
<b>Subject:</b> Re: [ClusterLabs] Two node cluster goes into split brain scenario during CPU intensive tasks<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Please, specify which you running and and check the cpu usage of your system, if you we are talking about user usage or system usage.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Best regards<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Il giorno dom 23 giu 2019 alle ore 13:40 Somanath Jeeva <<a href="mailto:somanath.jeeva@ericsson.com">somanath.jeeva@ericsson.com</a>> ha scritto:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">Hi All,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">I have a two node cluster with multicast (udp) transport . The multicast IP used in 224.1.1.1 .
</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">Whenever there is a CPU intensive task the pcs cluster goes into split brain scenario and doesn’t recover automatically . We have to do a manual restart of services to bring both nodes online again. Before the
nodes goes into split brain , the corosync log shows ,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Segoe UI",sans-serif">May 24 15:10:02 server1 corosync[4745]: [TOTEM ] Retransmit List: 7c 7e</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Segoe UI",sans-serif">May 24 15:10:02 server1 corosync[4745]: [TOTEM ] Retransmit List: 7c 7e</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Segoe UI",sans-serif">May 24 15:10:02 server1 corosync[4745]: [TOTEM ] Retransmit List: 7c 7e</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Segoe UI",sans-serif">May 24 15:10:02 server1 corosync[4745]: [TOTEM ] Retransmit List: 7c 7e</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Segoe UI",sans-serif">May 24 15:10:02 server1 corosync[4745]: [TOTEM ] Retransmit List: 7c 7e</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Segoe UI",sans-serif">May 24 15:51:42 server1 corosync[4745]: [TOTEM ] A processor failed, forming new configuration.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Segoe UI",sans-serif">May 24 16:41:42 server1 corosync[4745]: [TOTEM ] A new membership (<a href="http://10.241.31.12:29276" target="_blank">10.241.31.12:29276</a>) was formed. Members left: 1</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Segoe UI",sans-serif">May 24 16:41:42 server1 corosync[4745]: [TOTEM ] Failed to receive the leave message. failed: 1</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">Is there any way we can overcome this or this may be due to any multicast issues in the network side.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">With Regards</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">Somanath Thilak J</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
Manage your subscription:<br>
<a href="https://lists.clusterlabs.org/mailman/listinfo/users" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
<br>
ClusterLabs home: <a href="https://www.clusterlabs.org/" target="_blank">https://www.clusterlabs.org/</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<p class="MsoNormal"> .~.<br>
/V\<br>
// \\<br>
/( )\<br>
^`~'^<o:p></o:p></p>
</div>
</div>
</body>
</html>