<html><header></header><body><div style="font-family: Tahoma; font-size: 14px; color: #000000;">Reading sbd from SuSE I saw that it requires a special block to write informations, I don't think this is possibile here.</div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;"> </div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">It's a dual node ZFS storage running our own XStreamOS/illumos distribution, and here we're trying to add HA capabilities.</div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">We can move IPs, ZFS Pools and COMSTAR/iSCSI/FC, and now looking for a stable way to manage stonith.</div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;"> </div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">The hardware system is this:</div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;"> </div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;"><a href="https://www.supermicro.com/products/system/1u/1029/SYS-1029TP-DC0R.cfm">https://www.supermicro.com/products/system/1u/1029/SYS-1029TP-DC0R.cfm</a></div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;"> </div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">and it features a shared SAS3 backplane, so both nodes can see all the discs concurrently.</div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;"> </div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">Gabriele<br /><br />
<div id="wt-mailcard">
<div> </div>
<div> </div>
<div><span style="font-size: 14px; font-family: Helvetica;"><strong>Sonicle S.r.l. </strong>: <a href="http://www.sonicle.com/" target="_new">http://www.sonicle.com</a></span></div>
<div><span style="font-size: 14px; font-family: Helvetica;"><strong>Music: </strong><a href="http://www.gabrielebulfon.com/" target="_new">http://www.gabrielebulfon.com</a></span></div>
<div><span style="font-size: 14px; font-family: Helvetica;"><strong>Quantum Mechanics : </strong><a href="http://www.cdbaby.com/cd/gabrielebulfon" target="_new">http://www.cdbaby.com/cd/gabrielebulfon</a></span></div>
</div>
<br /><hr /><br /><br /><span style="font-family: Arial, Helvetica, sans-serif; font-size: small;"><strong>Da:</strong> Reid Wahl <nwahl@redhat.com><br /><strong>A:</strong> Cluster Labs - All topics related to open-source clustering welcomed <users@clusterlabs.org><br /><strong>Data:</strong> 30 luglio 2020 6.38.58 CEST<br /><strong>Oggetto:</strong> Re: [ClusterLabs] Antw: [EXT] Stonith failing<br /></span><br /><br />
<blockquote style="border-left: #000080 2px solid; margin-left: 5px; padding-left: 5px;">
<div dir="ltr">
<div>I don't know of a stonith method that acts upon a filesystem directly. You'd generally want to act upon the power state of the node or upon the underlying shared storage.</div>
<div> </div>
<div>What kind of hardware or virtualization platform are these systems running on? If there is a hardware watchdog timer, then sbd is possible. The fence_sbd agent (poison-pill fencing via block device) requires shared block storage, but sbd itself only requires a hardware watchdog timer.</div>
<div> </div>
<div>Additionally, there may be an existing fence agent that can connect to the controller you mentioned. What kind of controller is it?</div>
</div>
<br />
<div class="gmail_quote">
<div class="gmail_attr" dir="ltr">On Wed, Jul 29, 2020 at 5:24 AM Gabriele Bulfon <<a href="mailto:gbulfon@sonicle.com">gbulfon@sonicle.com</a>> wrote:</div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left: 1px solid #cccccc; padding-left: 1ex;">
<div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">Thanks a lot for the extensive explanation!</div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">Any idea about a ZFS stonith?</div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;"> </div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">Gabriele<br /><br />
<div id="gmail-m_-7185733964613104492wt-mailcard">
<div> </div>
<div> </div>
<div><span style="font-size: 14px; font-family: Helvetica;"><strong>Sonicle S.r.l. </strong>: <a href="http://www.sonicle.com/" target="_blank">http://www.sonicle.com</a></span></div>
<div><span style="font-size: 14px; font-family: Helvetica;"><strong>Music: </strong><a href="http://www.gabrielebulfon.com/" target="_blank">http://www.gabrielebulfon.com</a></span></div>
<div><span style="font-size: 14px; font-family: Helvetica;"><strong>Quantum Mechanics : </strong><a href="http://www.cdbaby.com/cd/gabrielebulfon" target="_blank">http://www.cdbaby.com/cd/gabrielebulfon</a></span></div>
</div>
<br /><hr /><br /><br /><span style="font-family: Arial,Helvetica,sans-serif; font-size: small;"><strong>Da:</strong> Reid Wahl <<a href="mailto:nwahl@redhat.com" target="_blank">nwahl@redhat.com</a>><br /><strong>A:</strong> Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org" target="_blank">users@clusterlabs.org</a>><br /><strong>Data:</strong> 29 luglio 2020 11.39.35 CEST<br /><strong>Oggetto:</strong> Re: [ClusterLabs] Antw: [EXT] Stonith failing<br /></span><br /><br />
<blockquote style="border-left: 2px solid #000080; margin-left: 5px; padding-left: 5px;">
<div dir="ltr">
<div>"As it stated in the comments, we don't want to halt or boot via ssh, only reboot."</div>
<div> </div>
<div>Generally speaking, a stonith reboot action consists of the following basic sequence of events:</div>
<div>
<ol>
<li>Execute the fence agent with the "off" action.</li>
<li>Poll the power status of the fenced node until it is powered off.</li>
<li>Execute the fence agent with the "on" action.</li>
<li>Poll the power status of the fenced node until it is powered on.</li>
</ol>
<div>So a custom fence agent that supports reboots, actually needs to support off and on actions.</div>
<div> </div>
<div> </div>
<div>As Andrei noted, ssh is **not** a reliable method by which to ensure a node gets rebooted or stops using cluster-managed resources. You can't depend on the ability to SSH to an unhealthy node that needs to be fenced.</div>
<div> </div>
<div>The only way to guarantee that an unhealthy or unresponsive node stops all access to shared resources is to power off or reboot the node. (In the case of resources that rely on shared storage, I/O fencing instead of power fencing can also work, but that's not ideal.)</div>
<div> </div>
<div>As others have said, SBD is a great option. Use it if you can. There are also power fencing methods (one example is fence_ipmilan, but the options available depend on your hardware or virt platform) that are reliable under most circumstances.</div>
<div> </div>
<div>You said that when you stop corosync on node 2, Pacemaker tries to fence node 2. There are a couple of possible reasons for that. One possibility is that you stopped or killed corosync without stopping Pacemaker first. (If you use pcs, then try `pcs cluster stop`.) Another possibility is that resources failed to stop during cluster shutdown on node 2, causing node 2 to be fenced.</div>
</div>
</div>
<br />
<div class="gmail_quote">
<div class="gmail_attr" dir="ltr">On Wed, Jul 29, 2020 at 12:47 AM Andrei Borzenkov <<a href="mailto:arvidjaar@gmail.com" target="_blank">arvidjaar@gmail.com</a>> wrote:</div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left: 1px solid #cccccc; padding-left: 1ex;">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default" style="font-family: arial,sans-serif;"> </div>
</div>
<br />
<div class="gmail_quote">
<div class="gmail_attr" dir="ltr">On Wed, Jul 29, 2020 at 9:01 AM Gabriele Bulfon <<a href="mailto:gbulfon@sonicle.com" target="_blank">gbulfon@sonicle.com</a>> wrote:</div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left: 1px solid #cccccc; padding-left: 1ex;">
<div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">That one was taken from a specific implementation on Solaris 11.</div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">The situation is a dual node server with shared storage controller: both nodes see the same disks concurrently.</div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">Here we must be sure that the two nodes are not going to import/mount the same zpool at the same time, or we will encounter data corruption:</div>
</div>
</blockquote>
<div> </div>
<div>
<div class="gmail_default" style="font-family: arial,sans-serif;">ssh based "stonith" cannot guarantee it.</div>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left: 1px solid #cccccc; padding-left: 1ex;">
<div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">node 1 will be perferred for pool 1, node 2 for pool 2, only in case one of the node goes down or is taken offline the resources should be first free by the leaving node and taken by the other node.</div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;"> </div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;">Would you suggest one of the available stonith in this case?</div>
<div style="font-family: Tahoma; font-size: 14px; color: #000000;"> </div>
</div>
</blockquote>
<div> </div>
<div>
<div class="gmail_default" style="font-family: arial,sans-serif;">IPMI, managed PDU, SBD ...</div>
</div>
<div class="gmail_default" style="font-family: arial,sans-serif;">In practice, the only stonith method that works in case of complete node outage including any power supply is SBD.</div>
</div>
</div>
_______________________________________________<br /> Manage your subscription:<br /> <a href="https://lists.clusterlabs.org/mailman/listinfo/users" target="_blank" rel="noreferrer">https://lists.clusterlabs.org/mailman/listinfo/users</a><br /> <br /> ClusterLabs home: <a href="https://www.clusterlabs.org/" target="_blank" rel="noreferrer">https://www.clusterlabs.org/</a></blockquote>
</div>
<br clear="all" /><br />-- <br />
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div>Regards,<br /><br /></div>
Reid Wahl, RHCA</div>
<div>Software Maintenance Engineer, Red Hat</div>
CEE - Platform Support Delivery - ClusterHA</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<pre>_______________________________________________
Manage your subscription:
<a href="https://lists.clusterlabs.org/mailman/listinfo/users" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a>
ClusterLabs home: <a href="https://www.clusterlabs.org/" target="_blank">https://www.clusterlabs.org/</a>
</pre>
</blockquote>
</div>
</div>
_______________________________________________<br /> Manage your subscription:<br /> <a href="https://lists.clusterlabs.org/mailman/listinfo/users" target="_blank" rel="noreferrer">https://lists.clusterlabs.org/mailman/listinfo/users</a><br /> <br /> ClusterLabs home: <a href="https://www.clusterlabs.org/" target="_blank" rel="noreferrer">https://www.clusterlabs.org/</a></blockquote>
</div>
<br clear="all" /><br />-- <br />
<div class="gmail_signature" dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div>Regards,<br /><br /></div>
Reid Wahl, RHCA</div>
<div>Software Maintenance Engineer, Red Hat</div>
CEE - Platform Support Delivery - ClusterHA</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<pre>_______________________________________________
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users
ClusterLabs home: https://www.clusterlabs.org/
</pre>
</blockquote>
</div></body></html>