<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jan 26, 2023 at 7:39 AM Thomas CAS <<a href="mailto:tcas@ikoula.com">tcas@ikoula.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg1572819650113939441">
<div style="overflow-wrap: break-word;" lang="FR">
<div class="m_-1836375398557362329WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hello,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">I'm having trouble with a MariaDB cluster (2 nodes, master-slave) on Debian 11.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">I don't know what to do anymore.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US">Environment:<u></u><u></u></span></b></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Node1: <u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> OS: Debian 11<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Kernel: 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Versions: resource-agents (4.7.0-1), pacemaker (2.0.5-2), corosync (3.1.2-2), mariadb (10.5.18-0+deb11u1)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Node2: <u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> OS: Debian 11<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Kernel: 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Versions: resource-agents (4.7.0-1), pacemaker (2.0.5-2), corosync (3.1.2-2), mariadb (10.5.18-0+deb11u1)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">crm configure show as attachment.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US">Problem: <u></u><u></u></span></b></p>
<p class="MsoNormal"><b><span lang="EN-US"><u></u> <u></u></span></b></p>
<p class="MsoNormal"><span lang="EN-US">When I restart Node2 (which is a slave), it goes up correctly in the cluster:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">$ crm status<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Cluster Summary:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> * Stack: corosync<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> * Current DC: Node1 (version 2.0.5-ba59be7122) - partition with quorum<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> * Last updated: Thu Jan 26 12:04:57 2023<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> * Last change: Thu Jan 26 11:39:58 2023 by root via cibadmin on Node2<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> * 2 nodes configured<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> * 3 resource instances configured<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Node List:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> * Online: [ Node1 Node2 ]<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Full List of Resources:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> * VIP (ocf::heartbeat:IPaddr2): Started Node1<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> * Clone Set: MYSQLREPLICATOR [MYSQL] (promotable):<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> * Masters: [ Node1 ]<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> * Slaves: [ Node2 ]<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">But it does not retrieve the replication information. (SHOW SLAVE STATUS; returns nothing)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">In the Node2 logs, I can see this message that explains that replication is not taking place:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<table style="border-collapse:collapse;border:medium none" cellspacing="0" cellpadding="0" border="1">
<tbody>
<tr>
<td style="width:469.8pt;border:1pt solid windowtext;background:black;padding:0cm 5.4pt" width="626" valign="top">
<p class="MsoNormal"><span style="color:white" lang="EN-US">Jan 25 16:29:38 mysql(MYSQL)[22862]: INFO: No MySQL master present - clearing replication state</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US">Jan 25 16:29:39 mysql(MYSQL)[22862]: WARNING: MySQL Slave IO threads currently not running.</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US">Jan 25 16:29:39 mysql(MYSQL)[22862]: ERROR: MySQL Slave SQL threads currently not running.</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US">Jan 25 16:29:39 mysql(MYSQL)[22862]: ERROR: See for details</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US">Jan 25 16:29:39 mysql(MYSQL)[22862]: ERROR: ERROR 1200 (HY000) at line 1: Misconfigured slave: MASTER_HOST was not set; Fix in config file or with CHANGE MASTER TO</span><span lang="EN-US"><u></u><u></u></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">From what I see in the following file, Node2 does not seem to find the master name. So it clears its replication information:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">/usr/lib/ocf/resource.d/heartbeat/mysql<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<table style="border-collapse:collapse;border:medium none" cellspacing="0" cellpadding="0" border="1">
<tbody>
<tr>
<td style="width:469.8pt;border:1pt solid windowtext;background:black;padding:0cm 5.4pt" width="626" valign="top">
<p class="MsoNormal"><span style="color:white" lang="EN-US"> master_host=`echo $OCF_RESKEY_CRM_meta_notify_master_uname|tr -d " "`</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> if [ "$master_host" -a "$master_host" != ${NODENAME} ]; then</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> ocf_log info "Changing MySQL configuration to replicate from $master_host."</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> set_master</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> start_slave</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> if [ $? -ne 0 ]; then</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> ocf_exit_reason "Failed to start slave"</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> return $OCF_ERR_GENERIC</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> fi</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> else</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> ocf_log info "No MySQL master present - clearing replication state"</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> unset_master</span><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:white" lang="EN-US"> fi</span><span lang="EN-US"><u></u><u></u></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">As it is a production environment, I performed a bare metal restore of these machines on 2 LAB machines and I have no problem…<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">In production, there is a lot of writing but the servers are far from being saturated.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Thank you in advance for all the help you can give me.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><br>
Best regards,</span></p></div></div></div></blockquote><div><br></div><div>I'm sorry you've encountered this.</div><div><br></div><div>I don't understand why the resource agent checks $OCF_RESKEY_CRM_meta_notify_master_uname during the start operation. That value gets set only during a notify operation. That looks like a bug in the resource agent.</div><div><br></div><div>I've filed an issue against it here: <a href="https://github.com/ClusterLabs/resource-agents/issues/1839">https://github.com/ClusterLabs/resource-agents/issues/1839</a></div><div><br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg1572819650113939441"><div style="overflow-wrap: break-word;" lang="FR"><div class="m_-1836375398557362329WordSection1"><p class="MsoNormal"><span lang="EN-US"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<table style="background:white;border-collapse:collapse" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="padding:0cm">
<p class="MsoNormal" style="line-height:18.75pt"><span style="font-size:13pt;color:rgb(30,25,90)">Thomas Cas</span><span style="color:rgb(60,86,106)"> </span><span style="font-size:13pt;color:rgb(255,120,40)"> | Technicien
du support infogérance</span><span style="color:rgb(60,86,106)"><u></u><u></u></span></p>
</td>
</tr>
<tr>
<td style="padding:0cm">
<p class="MsoNormal" style="line-height:18.75pt"><span style="font-size:10pt;color:rgb(30,25,90)" lang="EN-US">PHONE : </span><span style="color:black" lang="EN-US">+33 3 51 25 23 26
</span><span style="font-size:10pt;color:rgb(0,32,96)" lang="EN-US">WEB :</span><span style="color:black" lang="EN-US"> </span><span style="color:black"><a href="https://www.ikoula.com/en" target="_blank"><span style="color:black" lang="EN-US">www.ikoula.com/en</span></a></span><span style="font-size:10pt;color:rgb(17,17,17)" lang="EN-US"><u></u><u></u></span></p>
</td>
</tr>
<tr>
<td style="padding:0cm">
<p class="MsoNormal" style="line-height:18.75pt"><span style="font-size:10pt;color:rgb(17,17,17)">IKOULA Data Center 34 rue Pont Assy - 51100 Reims - FRANCE</span><span><u></u><u></u></span></p>
</td>
</tr>
<tr>
<td style="padding:0cm">
<p class="MsoNormal" style="line-height:21pt"><span style="font-size:10pt;color:rgb(51,102,0)" lang="EN-US">Before printing this letter, think about the impact on the environment!</span><span style="color:rgb(60,86,106)" lang="EN-US"><u></u><u></u></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<table style="background:white;border-collapse:collapse" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="padding:0cm" valign="bottom">
<p class="MsoNormal"><span style="color:black"><a href="https://www.ikoula.com/en" target="_blank"><span style="color:rgb(66,139,202);text-decoration:none"><img style="width: 2.7916in; height: 0.9479in;" id="m_-1836375398557362329Image_x0020_1" src="cid:185ef7bedf04cff311" alt="Ikoula" width="268" height="91" border="0"></span></a></span><span style="color:black"> </span><span><u></u><u></u></span></p>
</td>
<td style="padding:0cm" valign="bottom">
<p class="MsoNormal" style="margin-bottom:22.5pt"><span style="color:black"><a href="https://twitter.com/ikoula_en" title="Twitter Ikoula" target="_blank"><span style="color:rgb(66,139,202);text-decoration:none"><img style="width: 0.25in; height: 0.25in;" id="m_-1836375398557362329Image_x0020_2" src="cid:185ef7bedf15b16b22" alt="Twitter" width="24" height="24" border="0"></span></a></span><span style="color:rgb(60,86,106)"> </span><span style="color:black"><a href="https://www.linkedin.com/company/ikoula" title="Linkedin Ikoula" target="_blank"><span style="color:rgb(66,139,202);text-decoration:none"><img style="width: 0.25in; height: 0.25in;" id="m_-1836375398557362329Image_x0020_3" src="cid:185ef7bedf1692e333" alt="Linkedin" width="24" height="24" border="0"></span></a></span><span style="color:rgb(60,86,106)"> </span><span style="color:black"><a href="http://www.youtube.fr/ikoulanet" title="YouTube Ikoula" target="_blank"><span style="color:rgb(66,139,202);text-decoration:none"><img style="width: 0.25in; height: 0.25in;" id="m_-1836375398557362329Image_x0020_4" src="cid:185ef7bedf17745b44" alt="Youtube" width="24" height="24" border="0"></span></a></span><span style="color:rgb(60,86,106)"> </span><span style="color:black"><a href="https://pressroom.ikoula.com/" title="Pressroom Ikoula" target="_blank"><span style="color:rgb(66,139,202);text-decoration:none"><img style="width: 0.25in; height: 0.25in;" id="m_-1836375398557362329Image_x0020_5" src="cid:185ef7bedf1855d355" alt="Pressroom" width="24" height="24" border="0"></span></a></span><span style="color:rgb(60,86,106)"> </span><span style="color:black"><a href="https://blog.ikoula.com/en" title="Blog Ikoula" target="_blank"><span style="color:rgb(66,139,202);text-decoration:none"><img style="width: 0.25in; height: 0.25in;" id="m_-1836375398557362329Image_x0020_6" src="cid:185ef7bedf29374b66" alt="Blog" width="24" height="24" border="0"></span></a></span><span style="color:rgb(60,86,106)"> <u></u><u></u></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
_______________________________________________<br>
Manage your subscription:<br>
<a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
<br>
ClusterLabs home: <a href="https://www.clusterlabs.org/" rel="noreferrer" target="_blank">https://www.clusterlabs.org/</a><br>
</div></blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div>Regards,<br><br></div>Reid Wahl (He/Him)<br></div><div>Senior Software Engineer, Red Hat<br></div>RHEL High Availability - Pacemaker<br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>