<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Windl,</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Just a reminder about pacemaker 1.1.23 suspicious behaviour.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Would you please find time to check strace below and maybe forward me to a knowledgeable person.</div>
<div id="Signature" class="elementToProof">
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<p class="elementToProof" style="margin: 0in; font-family: Calibri, sans-serif; font-size: 11pt;">
Regards, Dmytro</p>
<p class="elementToProof" style="margin: 0in; font-family: Calibri, sans-serif; font-size: 11pt;">
</p>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Dmytro Poliarush <Dmytro_Poliarush@epam.com><br>
<b>Sent:</b> 25 March 2026 17:19<br>
<b>To:</b> Windl, Ulrich <u.windl@ukr.de>; Cluster Labs - All topics related to open-source clustering welcomed <users@clusterlabs.org><br>
<b>Subject:</b> Re: pacemaker: 1.1.23 20sec timeout on cluster with disc I/O write delays</font>
<div> </div>
</div>
<style type="text/css" style="display:none">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hi Windl,</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Thank yo very much for your prompt reply and links to timeout.</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I've tried all of those already and they are NOT working.</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
From my observation there is some kind of hardcoded 20sec timeout in stonithd on pacemaker 1.1.23.</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
In this pacemaker version stonithd is compled from: `commands.c`, `internal.h`, `main.c`, `remote.c`</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
And we assume that 20sec timeout is hardcoded somewhere in these sources.</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Most logical candidate so far was:</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
```</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
fencing/commands.c</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
#define DEFAULT_QUERY_TIMEOUT 20</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
```</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
But changing that value to 120 did NOT work.</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
strace still shows stonithd closing socket with stonith_admin after 20sec polling.</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
This is visible in attached: st_admin_strace.9964.comments.log:</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
```</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:08.680800 socket(AF_UNIX, SOCK_STREAM, 0) = 5<UNIX:[420071]> <0.000446></div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:08.694409 connect(5<UNIX:[420071]>, {sa_family=AF_UNIX, sun_path=@"stonith-ng\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"},
110) = 0 <0.000810></div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:08.699454 poll([{fd=5<UNIX:[420071->420078]>, events=POLLIN}], 1, 0) = 0 (Timeout) <0.000004></div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:08.699719 poll([{fd=5<UNIX:[420071->420078]>, events=POLLIN}], 1, 0) = 0 (Timeout) <0.000005></div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
... more polling on fd=5 here ...</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:08.700324 poll([{fd=5<UNIX:[420071->420078]>, events=POLLIN}], 1, 0) = 0 (Timeout) <0.000006></div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:09.099605 poll([{fd=5<UNIX:[420071->420078]>, events=POLLIN}], 1, 0) = 1 ([{fd=5, revents=POLLIN}]) <0.000092></div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:29.344300 shutdown(5<UNIX:[420071->420078]>, SHUT_RDWR) = 0 <0.000022></div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:29.344391 close(5<UNIX:[420071->420078]>) = 0 <0.000030></div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:29.346138 exit_group(-62) = ?</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:29.347107 +++ exited with 194 +++</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
```</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
What is MOST important here that strace top log shows stonith_admin started with `-t 60` (timeout in seconds).</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:08.659305 execve("/usr/sbin/stonith_admin", ["stonith_admin", "-VVV", "-t", "60", "-B", "node2"], 0x7ffd2b75e158 /* 33 vars */) = 0 <0.000162></div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
But somehow stonithd ignores that and after 20sec polling:</div>
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
05:45:29.344300 shutdown(5<UNIX:[420071->420078]>, SHUT_RDWR) = 0 <0.000022></div>
<div id="x_Signature" class="x_elementToProof">
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<p class="x_elementToProof" style="margin:0in; font-family:Calibri,sans-serif; font-size:11pt">
Regards, Dmytro</p>
<p class="x_elementToProof" style="margin:0in; font-family:Calibri,sans-serif; font-size:11pt">
</p>
</div>
<div id="x_appendonsend"></div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Windl, Ulrich <u.windl@ukr.de><br>
<b>Sent:</b> 23 March 2026 12:49<br>
<b>To:</b> Cluster Labs - All topics related to open-source clustering welcomed <users@clusterlabs.org><br>
<b>Cc:</b> Dmytro Poliarush <Dmytro_Poliarush@epam.com><br>
<b>Subject:</b> RE: pacemaker: 1.1.23 20sec timeout on cluster with disc I/O write delays</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
@font-face
{font-family:Aptos}
p.x_x_MsoNormal, li.x_x_MsoNormal, div.x_x_MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif}
.x_x_MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:70.85pt 70.85pt 2.0cm 70.85pt}
ol
{margin-bottom:0cm}
ul
{margin-bottom:0cm}
-->
</style>
<div lang="DE" style="word-wrap:break-word">
<table border="0" cellspacing="0" cellpadding="0" width="100%" align="left" style="background:revert!important; border:revert!important; bottom:revert!important; color:revert!important; direction:revert!important; display:revert!important; font-size:revert!important; height:revert!important; letter-spacing:revert!important; line-height:revert!important; margin:revert!important; opacity:revert!important; order:revert!important; outline:revert!important; overflow:revert!important; padding:revert!important; position:revert!important; tab-size:revert!important; table-layout:revert!important; text-align:revert!important; text-indent:revert!important; text-orientation:revert!important; text-overflow:revert!important; text-transform:revert!important; top:revert!important; vertical-align:revert!important; visibility:revert!important; white-space:revert!important; width:revert!important; word-break:revert!important; word-spacing:revert!important; writing-mode:revert!important; zoom:revert!important; border:0!important; display:table!important; width:100%!important; table-layout:fixed!important; border-collapse:seperate!important; float:none!important; border-spacing:0px 0px!important">
<tbody style="background:revert!important; border:revert!important; bottom:revert!important; color:revert!important; direction:revert!important; display:revert!important; font-size:revert!important; height:revert!important; letter-spacing:revert!important; line-height:revert!important; margin:revert!important; opacity:revert!important; order:revert!important; outline:revert!important; overflow:revert!important; padding:revert!important; position:revert!important; tab-size:revert!important; table-layout:revert!important; text-align:revert!important; text-indent:revert!important; text-orientation:revert!important; text-overflow:revert!important; text-transform:revert!important; top:revert!important; vertical-align:revert!important; visibility:revert!important; white-space:revert!important; width:revert!important; word-break:revert!important; word-spacing:revert!important; writing-mode:revert!important; zoom:revert!important; display:block!important">
<tr style="background:revert!important; border:revert!important; bottom:revert!important; color:revert!important; direction:revert!important; display:revert!important; font-size:revert!important; height:revert!important; letter-spacing:revert!important; line-height:revert!important; margin:revert!important; opacity:revert!important; order:revert!important; outline:revert!important; overflow:revert!important; padding:revert!important; position:revert!important; tab-size:revert!important; table-layout:revert!important; text-align:revert!important; text-indent:revert!important; text-orientation:revert!important; text-overflow:revert!important; text-transform:revert!important; top:revert!important; vertical-align:revert!important; visibility:revert!important; white-space:revert!important; width:revert!important; word-break:revert!important; word-spacing:revert!important; writing-mode:revert!important; zoom:revert!important">
<td valign="middle" width="1px" bgcolor="#A6A6A6" cellpadding="7px 2px 7px 2px" style="background:revert!important; border:revert!important; bottom:revert!important; color:revert!important; direction:revert!important; display:revert!important; font-size:revert!important; height:revert!important; letter-spacing:revert!important; line-height:revert!important; margin:revert!important; opacity:revert!important; order:revert!important; outline:revert!important; overflow:revert!important; padding:revert!important; position:revert!important; tab-size:revert!important; table-layout:revert!important; text-align:revert!important; text-indent:revert!important; text-orientation:revert!important; text-overflow:revert!important; text-transform:revert!important; top:revert!important; vertical-align:revert!important; visibility:revert!important; white-space:revert!important; width:revert!important; word-break:revert!important; word-spacing:revert!important; writing-mode:revert!important; zoom:revert!important; padding:7px 2px 7px 2px!important; background-color:#A6A6A6!important; width:0px!important">
</td>
<td valign="middle" width="100%" bgcolor="#EAEAEA" cellpadding="7px 5px 7px 15px" color="#212121" style="background:revert!important; border:revert!important; bottom:revert!important; color:revert!important; direction:revert!important; display:revert!important; font-size:revert!important; height:revert!important; letter-spacing:revert!important; line-height:revert!important; margin:revert!important; opacity:revert!important; order:revert!important; outline:revert!important; overflow:revert!important; padding:revert!important; position:revert!important; tab-size:revert!important; table-layout:revert!important; text-align:revert!important; text-indent:revert!important; text-orientation:revert!important; text-overflow:revert!important; text-transform:revert!important; top:revert!important; vertical-align:revert!important; visibility:revert!important; white-space:revert!important; width:revert!important; word-break:revert!important; word-spacing:revert!important; writing-mode:revert!important; zoom:revert!important; width:100%!important; background-color:#EAEAEA!important; padding:7px 5px 7px 15px!important; font-family:wf_segoe-ui_normal,Segoe UI,Segoe WP,Tahoma,Arial,sans-serif!important; font-size:12px!important; font-weight:normal!important; color:#212121!important; text-align:left!important; word-wrap:break-word!important">
<div style="background:revert!important; border:revert!important; bottom:revert!important; color:revert!important; direction:revert!important; display:revert!important; font-size:revert!important; height:revert!important; letter-spacing:revert!important; line-height:revert!important; margin:revert!important; opacity:revert!important; order:revert!important; outline:revert!important; overflow:revert!important; padding:revert!important; position:revert!important; tab-size:revert!important; table-layout:revert!important; text-align:revert!important; text-indent:revert!important; text-orientation:revert!important; text-overflow:revert!important; text-transform:revert!important; top:revert!important; vertical-align:revert!important; visibility:revert!important; white-space:revert!important; width:revert!important; word-break:revert!important; word-spacing:revert!important; writing-mode:revert!important; zoom:revert!important">
You don't often get email from u.windl@ukr.de. <a href="https://aka.ms/LearnAboutSenderIdentification" style="background:revert!important; color:revert!important; direction:revert!important; display:revert!important; font-size:revert!important; opacity:revert!important; visibility:revert!important">
Learn why this is important</a> </div>
</td>
<td valign="middle" align="left" width="75px" bgcolor="#EAEAEA" cellpadding="7px 5px 7px 5px" color="#212121" style="background:revert!important; border:revert!important; bottom:revert!important; color:revert!important; direction:revert!important; display:revert!important; font-size:revert!important; height:revert!important; letter-spacing:revert!important; line-height:revert!important; margin:revert!important; opacity:revert!important; order:revert!important; outline:revert!important; overflow:revert!important; padding:revert!important; position:revert!important; tab-size:revert!important; table-layout:revert!important; text-align:revert!important; text-indent:revert!important; text-orientation:revert!important; text-overflow:revert!important; text-transform:revert!important; top:revert!important; vertical-align:revert!important; visibility:revert!important; white-space:revert!important; width:revert!important; word-break:revert!important; word-spacing:revert!important; writing-mode:revert!important; zoom:revert!important; width:75px!important; background-color:#EAEAEA!important; padding:7px 5px 7px 5px!important; font-family:wf_segoe-ui_normal,Segoe UI,Segoe WP,Tahoma,Arial,sans-serif!important; font-size:12px!important; font-weight:normal!important; color:#212121!important; text-align:left!important; word-wrap:break-word!important">
</td>
</tr>
</tbody>
</table>
<div>
<div class="x_x_WordSection1">
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">I think you should provide more information, like SBD configuration, syslog messages, etc.</span></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Usually node fencing via SBD works by writing a message to a shared disk slot.</span></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Once written, SBD/pacemaker expects the node to suicide soon. However multiple timeouts are configurable.</span></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D"> </span></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Asking AI, I got this (treat with some care):</span></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">I'll search for the specific timeout parameters and their interdependencies in Linux Pacemaker clusters with SBD fencing.</span></p>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">SBD Timeout Parameters in Linux Pacemaker Clusters</span></b></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">In a Pacemaker cluster using SBD (STONITH Block Device) for fencing, there are
<b>four primary configurable timeout parameters</b>, and they have strict interdependencies.
</span><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Understanding these relationships is critical for reliable cluster operation.</span></p>
<div class="x_x_MsoNormal" align="center" style="text-align:center"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">
<hr size="1" width="100%" align="center">
</span></div>
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Timeout Parameters and Their Interdependencies</span></b></p>
<table class="x_x_MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="794" style="width:595.5pt; border-collapse:collapse">
<thead>
<tr>
<td valign="top" style="padding:0cm 12.0pt 3.0pt .75pt">
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Parameter</span></b></p>
</td>
<td valign="top" style="padding:0cm 12.0pt 3.0pt .75pt">
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Configuration Location</span></b></p>
</td>
<td valign="top" style="padding:0cm 12.0pt 3.0pt .75pt">
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Type</span></b></p>
</td>
<td valign="top" style="padding:0cm 12.0pt 3.0pt .75pt">
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Purpose</span></b></p>
</td>
<td valign="top" style="padding:0cm 12.0pt 3.0pt .75pt">
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Default</span></b></p>
</td>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">SBD_WATCHDOG_TIMEOUT</span></b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D"></span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">/etc/sysconfig/sbd</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">SBD daemon config</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Hardware watchdog timeout; triggers node self-fence if no kick received</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">5 seconds</span></p>
</td>
</tr>
<tr>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">msgwait</span></b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D"></span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">SBD device metadata</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">SBD device level</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Time window for message delivery to node slot on SBD device</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Set during device initialization</span></p>
</td>
</tr>
<tr>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">stonith-timeout</span></b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D"></span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Pacemaker CIB (cluster property)</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Global cluster property</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Maximum time Pacemaker waits for STONITH action (reboot/off) to complete</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">60 seconds</span></p>
</td>
</tr>
<tr>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">stonith-watchdog-timeout</span></b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D"></span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Pacemaker CIB (cluster property)</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Global cluster property</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Time Pacemaker assumes fencing has completed via watchdog (diskless SBD only)</span></p>
</td>
<td valign="top" style="padding:6.0pt 6.0pt 6.0pt .75pt">
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">0 (disabled by default)</span></p>
</td>
</tr>
</tbody>
</table>
<div class="x_x_MsoNormal" align="center" style="text-align:center"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">
<hr size="1" width="100%" align="center">
</span></div>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Critical Interdependencies</span></b></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">The timeout parameters have strict mathematical relationships that
<b>must be maintained</b> for proper cluster behavior:</span></p>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">For Disk-Based SBD (with shared storage devices):</span></b></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">msgwait >= (watchdog_timeout × 2)stonith-timeout >= msgwait + 20%</span></p>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Example:</span></b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D"> If watchdog timeout is
<b>30 seconds</b>:</span></p>
<ul type="disc" style="margin-top:0cm">
<li class="x_x_MsoNormal" style="color:#1F497D"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">msgwait must be at least <b>60 seconds</b></span></li><li class="x_x_MsoNormal" style="color:#1F497D"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">stonith-timeout must be at least <b>72 seconds</b> (60 + 20%)</span></li></ul>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">For Diskless SBD (watchdog-only, no shared storage):</span></b></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">stonith-watchdog-timeout >= (SBD_WATCHDOG_TIMEOUT × 2)stonith-timeout >= stonith-watchdog-timeout + 20%</span></p>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Example:</span></b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D"> If SBD_WATCHDOG_TIMEOUT is
<b>5 seconds</b>:</span></p>
<ul type="disc" style="margin-top:0cm">
<li class="x_x_MsoNormal" style="color:#1F497D"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">stonith-watchdog-timeout must be at least <b>10 seconds</b></span></li><li class="x_x_MsoNormal" style="color:#1F497D"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">stonith-timeout must be at least <b>12 seconds</b> (10 + 20%)</span></li></ul>
<div class="x_x_MsoNormal" align="center" style="text-align:center"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">
<hr size="1" width="100%" align="center">
</span></div>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">How These Parameters Interact</span></b></p>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Watchdog Timeout (SBD_WATCHDOG_TIMEOUT)</span></b></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">This is the
<b>foundation</b> of the timeout hierarchy. It represents how long the hardware watchdog will wait for a "kick" (heartbeat) from the SBD daemon before forcibly resetting the node. If storage latency or system issues prevent the SBD daemon from operating, the
node self-fences after this timeout expires.</span></p>
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Message Wait Timeout (msgwait)</span></b></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">This is set in the
<b>SBD device metadata</b> during initialization and defines the grace period for a fencing message to be acknowledged as delivered to the target node's slot. It must be
<b>at least twice the watchdog timeout</b> to ensure the node has time to detect the fencing message and self-fence gracefully before the watchdog triggers.</span></p>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">STONITH Timeout (stonith-timeout)</span></b></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">This is a
<b>Pacemaker cluster property</b> that controls how long the cluster waits for the fencing action to complete. It must exceed msgwait by at least
<b>20%</b> to allow sufficient time for the message to be delivered and processed. If this timeout is too short, the cluster may consider the fencing action failed and retry, causing unnecessary delays.</span></p>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">STONITH Watchdog Timeout (stonith-watchdog-timeout)</span></b></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">This parameter is
<b>only used for diskless SBD</b> and tells Pacemaker how long to wait before assuming a node has already self-fenced via the watchdog. It must be
<b>at least twice the SBD_WATCHDOG_TIMEOUT</b> to provide a safety margin. Setting this to
<b>0</b> (the default) disables resource recovery and is appropriate only for disk-based SBD configurations.</span></p>
<div class="x_x_MsoNormal" align="center" style="text-align:center"><span style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">
<hr size="1" width="100%" align="center">
</span></div>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Critical Warnings</span></b></p>
<p class="x_x_MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">Alice, pay attention to these constraints:</span></b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D"></span></p>
<ul type="disc" style="margin-top:0cm">
<li class="x_x_MsoNormal" style="color:#1F497D"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">Do not set stonith-watchdog-timeout until SBD is configured and running on every node</span></b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">, including Pacemaker Remote nodes.</span></li><li class="x_x_MsoNormal" style="color:#1F497D"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">If stonith-timeout < stonith-watchdog-timeout in diskless SBD</span></b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">, nodes can become stuck in an UNCLEAN state, blocking failover.</span></li><li class="x_x_MsoNormal" style="color:#1F497D"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">For multipath or iSCSI setups</span></b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">, the watchdog timeout should account for path failure detection and failover time. The max_polling_interval in /etc/multipath.conf must be less than the watchdog timeout.</span></li><li class="x_x_MsoNormal" style="color:#1F497D"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">Changing watchdog timeout requires coordinating changes across all dependent timeouts</span></b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif"> to maintain the mathematical relationships.</span></li><li class="x_x_MsoNormal" style="color:#1F497D"><b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif">Storage latency is the primary driver</span></b><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif"> of watchdog timeout values; high-latency storage requires longer timeouts, which cascades into longer msgwait and stonith-timeout values.</span></li></ul>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D">The interdependency structure ensures that each timeout layer provides sufficient time for the layer below it to complete, preventing race conditions
and cluster deadlock scenarios.</span></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D"> </span></p>
<p class="x_x_MsoNormal"><span lang="EN-US" style="font-size:11.0pt; font-family:"Aptos",serif; color:#1F497D"> </span></p>
<div>
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#1F497D">Kind regards,</span></p>
<p class="x_x_MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri",sans-serif; color:#1F497D">Ulrich Windl</span></p>
</div>
<p class="x_x_MsoNormal"><span style=""> </span></p>
<div style="border:none; border-left:solid blue 1.5pt; padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0cm 0cm 0cm">
<p class="x_x_MsoNormal"><b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri",sans-serif"> Users <users-bounces@clusterlabs.org>
<b>On Behalf Of </b>Dmytro Poliarush via Users<br>
<b>Sent:</b> Tuesday, March 17, 2026 12:32 PM<br>
<b>To:</b> users@clusterlabs.org<br>
<b>Cc:</b> Dmytro Poliarush <Dmytro_Poliarush@epam.com><br>
<b>Subject:</b> [EXT] [EXT] [ClusterLabs] pacemaker: 1.1.23 20sec timeout on cluster with disc I/O write delays</span></p>
</div>
</div>
<p class="x_x_MsoNormal"> </p>
<div style="border:solid black 1.0pt; padding:0cm 0cm 0cm 0cm; background:#FFF3CD">
<p style="background:#FFF3CD; border:none; padding:0cm"><span style="font-size:9.0pt; font-family:"Arial",sans-serif; color:black">Sicherheits-Hinweis: Diese E-Mail wurde von einer Person außerhalb des UKR gesendet. Seien Sie vorsichtig vor gefälschten Absendern,
wenn Sie auf Links klicken, Anhänge öffnen oder weitere Aktionen ausführen, bevor Sie die Echtheit überprüft haben.</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">Hi all,</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">Need a small guidance on pacemaker: 1.1.23.</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">I'm chasing a stubborn issue in a 2node 2disc SBD cluster.</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">When running manual fencing test with `pcs stonith fence` command, I observe an error</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">```</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> Error: unable to fence '<nodehostname>'</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">```</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">Error manifests each time around a `20second` timeout(I assume this is a timeout).</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">`time` command is used to track how long execution runs: `time pcs stonith fence`.</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">Here is an example:</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">```</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> [root@node1 ~]# time pcs stonith fence --debug node2</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> Running: /usr/sbin/stonith_admin -B node2</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> > Return Value: 194</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> --Debug Output Start--</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> --Debug Output End--</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> Error: unable to fence 'node2'</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> > real 0m20.791s</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> user 0m0.063s</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> sys 0m0.033s</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> [root@node1 ~]#</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">```</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">For investigation, I've setup a testing cluster with 2 Virtualbox VMs.</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">Behaviour was NOT observed on testing cluster until I intentionally added disk write delays with dmsetup tool on one of the nodes.</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">Here is an example of setting a 22sec write delay:</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">```</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> # Create: read delay = 0 ms, write delay = 22000 ms</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> # Table format: delay <dev> <start> <read_ms> <dev> <start> <write_ms></span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> dmsetup --noudevsync create slow-sdc --table "0 ${SIZE} delay /dev/sdc 0 0 /dev/sdc 0 22000"</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> dmsetup mknodes</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">```</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">NOTE, that tests with delays upto(including) 19sec pass:</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">```</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> [root@node1 ~]# ./suspend-resume-slow-sdc-delay-write.sh 20000</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> [root@node1 ~]# dmsetup table slow-sdc</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">> 0 262144 delay 8:32 0 0 8:32 0 20000</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> [root@node1 ~]# time pcs stonith fence --debug node2</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> Running: /usr/sbin/stonith_admin -B node2</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> Return Value: 194</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> --Debug Output Start--</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> --Debug Output End--</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">```</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">> Error: unable to fence 'node2'</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">> real 0m20.588s</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> user 0m0.088s</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> sys 0m0.021s</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">> [root@node1 ~]# ./suspend-resume-slow-sdc-delay-write.sh 19000</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> ++ blockdev --getsize /dev/sdc</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> + SIZE=262144</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> ++ lsblk -dn -o MAJ:MIN /dev/sdc</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> + MAJMIN=' 8:32 '</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> + dmsetup suspend slow-sdc</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> + dmsetup reload slow-sdc --table '0 262144 delay /dev/sdc 0 0 /dev/sdc 0 19000'</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> + dmsetup resume slow-sdc</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> + dmsetup table slow-sdc</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">> 0 262144 delay 8:32 0 0 8:32 0 19000</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> [root@node1 ~]# pcs stonith history cleanup; pcs stonith cleanup # pcs-cleanup-error-cleanup</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> cleaning up fencing-history for node *</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> Cleaned up all resources on all nodes</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> [root@node1 ~]#</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> [root@node1 ~]# time pcs stonith fence --debug node2</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> Running: /usr/sbin/stonith_admin -B node2</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> Return Value: 0</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> --Debug Output Start--</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> --Debug Output End--</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">> Node: node2 fenced</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">> real 0m19.869s</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> user 0m0.098s</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> sys 0m0.035s</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> [root@node1 ~]#</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">```</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">So here is my question:</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">I assume there is a 20sec timeout value hardcoded somewhere in pacemaker 1.1.23 sources.</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">This hardcoded value impacts manual fencing in case of disc I/O delays(maybe in some other cases).</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">I expect that increasing timeout can mitigate clusters with disc I/O issues similar to ones described above.</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">Please note this timeout is NOT: stonith-timeout or stonith-watchdog-timeout.</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">Could you please comment if that is a meaningfull assumption and where does the 20sec timeout come from.</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="font-family:"Aptos",serif; color:black">Regards, Dmytro</span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>