<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">And just to make sure, I’m not the kind of person who stick to the “we always did it that way…” ;)<div class="">Just trying to figure out why it suddenly breaks.<br class=""><div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class="Apple-interchange-newline">-derek</div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">--</div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Derek Wuelfrath</div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="mailto:dwuelfrath@inverse.ca" class="">dwuelfrath@inverse.ca</a> :: +1.514.447.4918 (x110) :: +1.866.353.6153 (x110)</div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Inverse inc. :: Leaders behind SOGo (<a href="https://www.sogo.nu" class="">www.sogo.nu</a>), PacketFence (<a href="https://www.packetfence.org/" class="">www.packetfence.org</a>) and Fingerbank (<a href="https://www.fingerbank.org" class="">www.fingerbank.org</a>)</div></div></div></div></div></div></div></div></div></div></div></div>
</div>
<div><br class=""><blockquote type="cite" class=""><div class="">On Nov 15, 2017, at 15:30, Derek Wuelfrath <<a href="mailto:dwuelfrath@inverse.ca" class="">dwuelfrath@inverse.ca</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">I agree. Thing is, we have this kind of setup deployed largely and since a while. Never ran into any issue.<div class="">Not sure if something changed in Corosync/Pacemaker code or way of dealing with systemd resources.</div><div class=""><br class=""></div><div class="">As said, without a systemd resource, everything just work as it should… 100% of the time</div><div class="">As soon as a systemd resource comes in, it breaks.<br class=""><div class="">
<div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class="Apple-interchange-newline">-derek</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">--</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Derek Wuelfrath</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="mailto:dwuelfrath@inverse.ca" class="">dwuelfrath@inverse.ca</a> :: +1.514.447.4918 (x110) :: +1.866.353.6153 (x110)</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-position: normal; font-variant-caps: normal; font-variant-numeric: normal; font-variant-alternates: normal; font-variant-east-asian: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Inverse inc. :: Leaders behind SOGo (<a href="https://www.sogo.nu/" class="">www.sogo.nu</a>), PacketFence (<a href="https://www.packetfence.org/" class="">www.packetfence.org</a>) and Fingerbank (<a href="https://www.fingerbank.org/" class="">www.fingerbank.org</a>)</div></div></div></div></div></div></div></div></div></div></div></div>
</div>
<div class=""><br class=""><blockquote type="cite" class=""><div class="">On Nov 14, 2017, at 23:03, Digimer <<a href="mailto:lists@alteeve.ca" class="">lists@alteeve.ca</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="moz-cite-prefix" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">Quorum doesn't prevent split-brains, stonith (fencing) does.<span class="Apple-converted-space"> </span><br class=""><br class=""><a class="moz-txt-link-freetext" href="https://www.alteeve.com/w/The_2-Node_Myth" style="color: purple; text-decoration: underline;">https://www.alteeve.com/w/The_2-Node_Myth</a><br class=""><br class="">There is no way to use quorum-only to avoid a potential split-brain. You might be able to make it less likely with enough effort, but never prevent it.<br class=""><br class="">digimer<br class=""><br class="">On 2017-11-14 10:45 PM, Garima wrote:<br class=""></div><blockquote type="cite" cite="mid:86CE92308A86944992E6D1B25BD638AEAE79734A@EXCH-MB01-DEL.nectechnologies.in" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class=""><div class="WordSection1" style="page: WordSection1;"><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Hello All,<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Split-brain situation occurs due to there is a drop in quorum which leads to Spilt-brain situation and status information is not exchanged between both two nodes of the cluster.<span class="Apple-converted-space"> </span><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">This can be avoided if quorum communicates between both the nodes.<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">I have checked the code. In My opinion these files need to be updated (quorum.py/stonith.py) to avoid the spilt-brain situation to maintain Active-Passive configuration.<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Regards,<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Garima<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div class=""><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;" class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><b class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">From:</span></b><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="Apple-converted-space"> </span>Derek Wuelfrath [<a class="moz-txt-link-freetext" href="mailto:dwuelfrath@inverse.ca" style="color: purple; text-decoration: underline;">mailto:dwuelfrath@inverse.ca</a>]<span class="Apple-converted-space"> </span><br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>13 November 2017 20:55<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span>Cluster Labs - All topics related to open-source clustering welcomed<span class="Apple-converted-space"> </span><a class="moz-txt-link-rfc2396E" href="mailto:users@clusterlabs.org" style="color: purple; text-decoration: underline;"><users@clusterlabs.org></a><br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [ClusterLabs] Pacemaker responsible of DRBD and a systemd resource<o:p class=""></o:p></span></div></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">Hello Ken !<o:p class=""></o:p></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">Make sure that the systemd service is not enabled. If pacemaker is<br class="">managing a service, systemd can't also be trying to start and stop it.<o:p class=""></o:p></div></blockquote><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">It is not. I made sure of this in the first place :)<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">Beyond that, the question is what log messages are there from around<br class="">the time of the issue (on both nodes).<o:p class=""></o:p></div></blockquote><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">Well, that’s the thing. There is not much log messages telling what is actually happening. The ’systemd’ resource is not even trying to start (nothing in either log for that resource). Here are the logs from my last attempt:<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">Scenario:<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">- Services were running on ‘pancakeFence2’. DRBD was synced and connected<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">- I rebooted ‘pancakeFence2’. Services failed to ‘pancakeFence1’<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">- After ‘pancakeFence2’ comes back, services are running just fine on ‘pancakeFence1’ but DRBD is in Standalone due to split-brain<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">Logs for pancakeFence1: <a href="https://pastebin.com/dVSGPP78" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">https://pastebin.com/dVSGPP78</a><o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">Logs for pancakeFence2: <a href="https://pastebin.com/at8qPkHE" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">https://pastebin.com/at8qPkHE</a><o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">It really looks like the status checkup mechanism of corosync/pacemaker for a systemd resource force the resource to “start” and therefore, start the ones above that resource in the group (DRBD in instance).<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">This does not happen for a regular OCF resource (IPaddr2 per example)<o:p class=""></o:p></div></div><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><br class="">Cheers!<o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">-dw<o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><o:p class=""> </o:p></span></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">--<o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">Derek Wuelfrath<o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><a href="mailto:dwuelfrath@inverse.ca" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">dwuelfrath@inverse.ca</a> :: +1.514.447.4918 (x110) :: +1.866.353.6153 (x110)<o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">Inverse inc. :: Leaders behind SOGo (<a href="https://www.sogo.nu/" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">www.sogo.nu</a>), PacketFence (<a href="https://www.packetfence.org/" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">www.packetfence.org</a>) and Fingerbank (<a href="https://www.fingerbank.org/" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">www.fingerbank.org</a>)<o:p class=""></o:p></span></div></div></div></div></div></div></div></div></div></div></div></div></div></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><br class=""><br class=""><o:p class=""></o:p></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class="">On Nov 10, 2017, at 11:39, Ken Gaillot <<a href="mailto:kgaillot@redhat.com" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">kgaillot@redhat.com</a>> wrote:<o:p class=""></o:p></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">On Thu, 2017-11-09 at 20:27 -0500, Derek Wuelfrath wrote:<br style="font-variant-caps: normal; text-align: start; -webkit-text-stroke-width: 0px; word-spacing: 0px;" class=""><br class=""></span><o:p class=""></o:p></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">Hello there,<br class=""><br class="">First post here but following since a while!<o:p class=""></o:p></span></div></blockquote><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><br class="">Welcome!<br class=""><br style="font-variant-caps: normal; text-align: start; -webkit-text-stroke-width: 0px; word-spacing: 0px;" class=""><br class=""></span><o:p class=""></o:p></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><br class="">Here’s my issue,<br class="">we are putting in place and running this type of cluster since a<br class="">while and never really encountered this kind of problem.<br class=""><br class="">I recently set up a Corosync / Pacemaker / PCS cluster to manage DRBD<br class="">along with different other resources. Part of theses resources are<br class="">some systemd resources… this is the part where things are “breaking”.<br class=""><br class="">Having a two servers cluster running only DRBD or DRBD with an OCF<br class="">ipaddr2 resource (Cluser IP in instance) works just fine. I can<br class="">easily move from one node to the other without any issue.<br class="">As soon as I add a systemd resource to the resource group, things are<br class="">breaking. Moving from one node to the other using standby mode works<br class="">just fine but as soon as Corosync / Pacemaker restart involves<br class="">polling of a systemd resource, it seems like it is trying to start<br class="">the whole resource group and therefore, create a split-brain of the<br class="">DRBD resource.<o:p class=""></o:p></span></div></blockquote><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><br class="">My first two suggestions would be:<br class=""><br class="">Make sure that the systemd service is not enabled. If pacemaker is<br class="">managing a service, systemd can't also be trying to start and stop it.<br class=""><br class="">Fencing is the only way pacemaker can resolve split-brains and certain<br class="">other situations, so that will help in the recovery.<br class=""><br class="">Beyond that, the question is what log messages are there from around<br class="">the time of the issue (on both nodes).<br class=""><br class=""><br style="font-variant-caps: normal; text-align: start; -webkit-text-stroke-width: 0px; word-spacing: 0px;" class=""><br class=""></span><o:p class=""></o:p></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><br class="">It is the best explanation / description of the situation that I can<br class="">give. If it need any clarification, examples, … I am more than open<br class="">to share them.<br class=""><br class="">Any guidance would be appreciated :)<br class=""><br class="">Here’s the output of a ‘pcs config’<br class=""><br class=""><a href="https://pastebin.com/1TUvZ4X9" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">https://pastebin.com/1TUvZ4X9</a><br class=""><br class="">Cheers!<br class="">-dw<br class=""><br class="">--<br class="">Derek Wuelfrath<br class=""><a href="mailto:dwuelfrath@inverse.ca" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">dwuelfrath@inverse.ca</a> :: +1.514.447.4918 (x110) :: +1.866.353.6153<br class="">(x110)<br class="">Inverse inc. :: Leaders behind SOGo (<a href="http://www.sogo.nu/" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">www.sogo.nu</a>), PacketFence<br class="">(<a href="http://www.packetfence.org/" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">www.packetfence.org</a>) and Fingerbank (<a href="http://www.fingerbank.org/" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class="">www.fingerbank.org</a>)<o:p class=""></o:p></span></div></blockquote><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">--<span class="apple-converted-space"> </span><br class="">Ken Gaillot <</span><a href="mailto:kgaillot@redhat.com" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">kgaillot@redhat.com</span></a><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">><br class=""><br class="">_______________________________________________<br class="">Users mailing list:<span class="apple-converted-space"> </span></span><a href="mailto:Users@clusterlabs.org" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">Users@clusterlabs.org</span></a><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><br class=""></span><a href="http://lists.clusterlabs.org/mailman/listinfo/users" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">http://lists.clusterlabs.org/mailman/listinfo/users</span></a><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><br class=""><br class="">Project Home:<span class="apple-converted-space"> </span></span><a href="http://www.clusterlabs.org/" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">http://www.clusterlabs.org</span></a><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><br class="">Getting started:<span class="apple-converted-space"> </span></span><a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</span></a><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class=""><br class="">Bugs:<span class="apple-converted-space"> </span></span><a href="http://bugs.clusterlabs.org/" moz-do-not-send="true" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">http://bugs.clusterlabs.org</span></a><o:p class=""></o:p></div></div></blockquote></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div></div><br class=""><fieldset class="mimeAttachmentHeader"></fieldset><br class=""><pre wrap="" class="">_______________________________________________
Users mailing list: <a class="moz-txt-link-abbreviated" href="mailto:Users@clusterlabs.org" style="color: purple; text-decoration: underline;">Users@clusterlabs.org</a>
<a class="moz-txt-link-freetext" href="http://lists.clusterlabs.org/mailman/listinfo/users" style="color: purple; text-decoration: underline;">http://lists.clusterlabs.org/mailman/listinfo/users</a>
Project Home: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org/" style="color: purple; text-decoration: underline;">http://www.clusterlabs.org</a>
Getting started: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" style="color: purple; text-decoration: underline;">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a>
Bugs: <a class="moz-txt-link-freetext" href="http://bugs.clusterlabs.org/" style="color: purple; text-decoration: underline;">http://bugs.clusterlabs.org</a>
</pre></blockquote><p style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class=""><br class=""></p><pre class="moz-signature" cols="72" style="font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">--
Digimer
Papers and Projects: <a class="moz-txt-link-freetext" href="https://alteeve.com/w/" style="color: purple; text-decoration: underline;">https://alteeve.com/w/</a>
"I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould</pre><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); float: none; display: inline !important;" class="">Users mailing list:<span class="Apple-converted-space"> </span></span><a href="mailto:Users@clusterlabs.org" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class="">Users@clusterlabs.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class=""><a href="http://lists.clusterlabs.org/mailman/listinfo/users" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class="">http://lists.clusterlabs.org/mailman/listinfo/users</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); float: none; display: inline !important;" class="">Project Home:<span class="Apple-converted-space"> </span></span><a href="http://www.clusterlabs.org/" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class="">http://www.clusterlabs.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); float: none; display: inline !important;" class="">Getting started:<span class="Apple-converted-space"> </span></span><a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class="">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); float: none; display: inline !important;" class="">Bugs:<span class="Apple-converted-space"> </span></span><a href="http://bugs.clusterlabs.org/" style="color: purple; text-decoration: underline; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class="">http://bugs.clusterlabs.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);" class=""></div></blockquote></div><br class=""></div></div>_______________________________________________<br class="">Users mailing list: <a href="mailto:Users@clusterlabs.org" class="">Users@clusterlabs.org</a><br class=""><a href="http://lists.clusterlabs.org/mailman/listinfo/users" class="">http://lists.clusterlabs.org/mailman/listinfo/users</a><br class=""><br class="">Project Home: http://www.clusterlabs.org<br class="">Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf<br class="">Bugs: http://bugs.clusterlabs.org<br class=""></div></blockquote></div><br class=""></div></body></html>