<html><head><meta http-equiv="content-type" content="text/html; charset=GB2312"><style>body { line-height: 1.5; }body { font-size: 14px; font-family: "Microsoft YaHei UI"; color: rgb(0, 0, 0); line-height: 1.5; }</style></head><body>
<div><span></span><br></div><div><br></div><div><div><font face="Arial">Hi all,</font></div><div><font face="Arial">I am writing to report an issue with uneven resource migration in our Lustre cluster. Below are the details:</font></div><div><font face="Arial"><br></font></div><div><font face="Arial"><b>一 Background</b>:</font></div><div><font face="Arial">We have 3 physical nodes, each hosting 2 virtual machines: lustre-mds-nodexx (containing 2 MDTs) and lustre-oss-nodexx (containing 8 OSTs and MGS on one of them).</font></div><div><font face="Arial">We are using Lustre version 2.15.5 along with Pacemaker(2.1.0) for cluster management.</font></div><div><font face="Arial"><br></font></div><div><font face="Arial"><b>二 Problem:</b></font></div><div><font face="Arial">After powering off lustre-oss-node144 using the command virsh destroy lustre-oss-node144, the resources from lustre-oss-node144 did not migrate evenly. All resources migrated to lustre-oss-node31.</font></div><div><font face="Arial"><br></font></div><div><font face="Arial"><b>三 Resource Status Before and After powering off lustre-oss-node144:</b></font></div><div><font face="Arial" style="font-size: 13px;"><b>Before :</b></font></div><div><font face="Arial">[root@lustre-oss-node31 ~]# pcs status</font></div><div><font face="Arial">Cluster name: oss_cluster</font></div><div><font face="Arial">Cluster Summary:</font></div><div><font face="Arial"> * Stack: corosync (Pacemaker is running)</font></div><div><font face="Arial"> * Current DC: lustre-oss-node144 (version 2.1.7-5.el8_10-0f7f88312) - partition with quorum</font></div><div><font face="Arial"> * Last updated: Fri Jun 6 14:10:54 2025 on lustre-oss-node31</font></div><div><font face="Arial"> * Last change: Fri Jun 6 14:06:46 2025 by root via root on lustre-oss-node31</font></div><div><font face="Arial"> * 3 nodes configured</font></div><div><font face="Arial"> * 28 resource instances configured</font></div><div><font face="Arial"><br></font></div><div><font face="Arial">Node List:</font></div><div><font face="Arial"> * Online: [ lustre-oss-node31 lustre-oss-node135 lustre-oss-node144 ]</font></div><div><font face="Arial"><br></font></div><div><font face="Arial">Full List of Resources:</font></div><div><font face="Arial"> * vmfence_lustre-oss-node31 (stonith:fence_xvm): Started lustre-oss-node144</font></div><div><font face="Arial"> * vmfence_lustre-oss-node144 (stonith:fence_xvm): Started lustre-oss-node135</font></div><div><font face="Arial"> * vmfence_lustre-oss-node135 (stonith:fence_xvm): Started lustre-oss-node31</font></div><div><font face="Arial"> * mgt (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-0 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-3 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-6 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-9 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-12 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-15 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-18 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-21 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-1 (ocf::heartbeat:Filesystem): Started lustre-oss-node144</font></div><div><font face="Arial"> * ost-4 (ocf::heartbeat:Filesystem): Started lustre-oss-node144</font></div><div><font face="Arial"> * ost-7 (ocf::heartbeat:Filesystem): Started lustre-oss-node144</font></div><div><font face="Arial"> * ost-10 (ocf::heartbeat:Filesystem): Started lustre-oss-node144</font></div><div><font face="Arial"> * ost-13 (ocf::heartbeat:Filesystem): Started lustre-oss-node144</font></div><div><font face="Arial"> * ost-16 (ocf::heartbeat:Filesystem): Started lustre-oss-node144</font></div><div><font face="Arial"> * ost-19 (ocf::heartbeat:Filesystem): Started lustre-oss-node144</font></div><div><font face="Arial"> * ost-22 (ocf::heartbeat:Filesystem): Started lustre-oss-node144</font></div><div><font face="Arial"> * ost-2 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-5 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-8 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-11 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-14 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-17 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-20 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-23 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"><br></font></div><div><font face="Arial"><b style="font-size: 13px;">2 After</b></font></div><div><font face="Arial">[root@lustre-oss-node31 ~]# date;pcs status</font></div><div><font face="Arial">Fri Jun 6 14:12:50 CST 2025</font></div><div><font face="Arial">Cluster name: oss_cluster</font></div><div><font face="Arial">Cluster Summary:</font></div><div><font face="Arial"> * Stack: corosync (Pacemaker is running)</font></div><div><font face="Arial"> * Current DC: lustre-oss-node135 (version 2.1.7-5.el8_10-0f7f88312) - partition with quorum</font></div><div><font face="Arial"> * Last updated: Fri Jun 6 14:12:50 2025 on lustre-oss-node31</font></div><div><font face="Arial"> * Last change: Fri Jun 6 14:06:46 2025 by root via root on lustre-oss-node31</font></div><div><font face="Arial"> * 3 nodes configured</font></div><div><font face="Arial"> * 28 resource instances configured</font></div><div><font face="Arial"><br></font></div><div><font face="Arial">Node List:</font></div><div><font face="Arial"> * Online: [ lustre-oss-node31 lustre-oss-node135 ]</font></div><div><font face="Arial"> * OFFLINE: [ lustre-oss-node144 ]</font></div><div><font face="Arial"><br></font></div><div><font face="Arial">Full List of Resources:</font></div><div><font face="Arial"> * vmfence_lustre-oss-node31 (stonith:fence_xvm): Started lustre-oss-node135</font></div><div><font face="Arial"> * vmfence_lustre-oss-node144 (stonith:fence_xvm): Started lustre-oss-node135</font></div><div><font face="Arial"> * vmfence_lustre-oss-node135 (stonith:fence_xvm): Started lustre-oss-node31</font></div><div><font face="Arial"> * mgt (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-0 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-3 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-6 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-9 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-12 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-15 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-18 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-21 (ocf::heartbeat:Filesystem): Started lustre-oss-node135</font></div><div><font face="Arial"> * ost-1 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-4 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-7 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-10 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-13 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-16 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-19 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-22 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-2 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-5 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-8 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-11 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-14 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-17 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-20 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"> * ost-23 (ocf::heartbeat:Filesystem): Started lustre-oss-node31</font></div><div><font face="Arial"><br></font></div><div><font face="Arial">Daemon Status:</font></div><div><font face="Arial"> corosync: active/enabled</font></div><div><font face="Arial"> pacemaker: active/enabled</font></div><div><font face="Arial"> pcsd: active/enabled</font></div><div><font face="Arial"><br></font></div><div><font face="Arial"><b>三 Logs:</b></font></div><div><font face="Arial">Jun 06 14:11:12 lustre-oss-node135 pacemaker-schedulerd[1069268] (log_list_item) <span style="white-space:pre"> </span>notice: Actions: Move ost-1 ( lustre-oss-node144 -> lustre-oss-node31 )</font></div><div><font face="Arial">Jun 06 14:11:12 lustre-oss-node135 pacemaker-schedulerd[1069268] (log_list_item) <span style="white-space:pre"> </span>notice: Actions: Move ost-4 ( lustre-oss-node144 -> lustre-oss-node31 )</font></div><div><font face="Arial">Jun 06 14:11:12 lustre-oss-node135 pacemaker-schedulerd[1069268] (log_list_item) <span style="white-space:pre"> </span>notice: Actions: Move ost-7 ( lustre-oss-node144 -> lustre-oss-node31 )</font></div><div><font face="Arial">Jun 06 14:11:12 lustre-oss-node135 pacemaker-schedulerd[1069268] (log_list_item) <span style="white-space:pre"> </span>notice: Actions: Move ost-10 ( lustre-oss-node144 -> lustre-oss-node31 )</font></div><div><font face="Arial">Jun 06 14:11:12 lustre-oss-node135 pacemaker-schedulerd[1069268] (log_list_item) <span style="white-space:pre"> </span>notice: Actions: Move ost-13 ( lustre-oss-node144 -> lustre-oss-node31 )</font></div><div><font face="Arial">Jun 06 14:11:12 lustre-oss-node135 pacemaker-schedulerd[1069268] (log_list_item) <span style="white-space:pre"> </span>notice: Actions: Move ost-16 ( lustre-oss-node144 -> lustre-oss-node31 )</font></div><div><font face="Arial">Jun 06 14:11:12 lustre-oss-node135 pacemaker-schedulerd[1069268] (log_list_item) <span style="white-space:pre"> </span>notice: Actions: Move ost-19 ( lustre-oss-node144 -> lustre-oss-node31 )</font></div><div><font face="Arial">Jun 06 14:11:12 lustre-oss-node135 pacemaker-schedulerd[1069268] (log_list_item) <span style="white-space:pre"> </span>notice: Actions: Move ost-22 ( lustre-oss-node144 -> lustre-oss-node31 )</font></div><div><font face="Arial"><br></font></div><div><font face="Arial"><b>四 Attachments:</b></font></div><div><font face="Arial">The attached files include the configuration(config.txt) and logs(node135.log) during the uneven migration.</font></div><div><font face="Arial"><br></font></div><div><font face="Arial">Thank you for your attention and support.</font></div><div><font face="Arial">Best regards</font></div></div><div><br></div><div><br></div><hr style="width: 210px; height: 1px;" color="#b5c4df" size="1" align="left">
<div><span><div style="MARGIN: 10px; FONT-FAMILY: verdana; FONT-SIZE: 10pt"><div>chenzufei@gmail.com</div></div></span></div>
</body></html>