[ClusterLabs] [EXT] How to delete a node, really, really?

[Kadlecsik József]

Hello,

On Thu, 12 Dec 2024, Kadlecsik József via Users wrote:

> On Thu, 12 Dec 2024, Windl, Ulrich wrote:
> 
> > Did you restart Corosync as well?
> 
> Yes. All nodes was put to standby mode one by one and then rebooted.
> 
> My last idea is to enable maintenace mode and stop pacemaker and corosync 
> on all nodes and then restart. But if there's a less intrusive mode to 
> remove the ghost nodes, I'd better use that.

Executing a full restart of pacemaker/corosync as above helped to cleanup 
the system and get rid of the ghost entries. Somehow crm_node and 
cibadmin were not enough.

Best regards,
Jozsef

> > > -----Original Message-----
> > > From: Users <users-bounces at clusterlabs.org> On Behalf Of Kadlecsik József
> > > via Users
> > > Sent: Thursday, December 12, 2024 12:04 PM
> > > To: users at clusterlabs.org
> > > Cc: Kadlecsik József <kadlecsik.jozsef at wigner.hu>
> > > Subject: [EXT] [ClusterLabs] How to delete a node, really, really?
> > > 
> > > Hello,
> > > 
> > > During an upgrade it turned out that our old node id-s (generated
> > > many-many years ago) are not accepted anymore. So we renumbered the
> > > nodes
> > > and since then have been unable to delete the old id-s from the system.
> > > 
> > > Corosync knows the new id-s only and work fine:
> > > 
> > > # corosync-cmapctl | grep 323224576
> > > #
> > > 
> > > Pacemaker knows the new id-s according to the config file and all traces
> > > are deleted from the cib as well:
> > > 
> > > # cibadmin --query|grep 323224576
> > > #
> > > 
> > > Everything seems ok. However, as soon as just corosync stopped or started
> > > on a standby node, the old ghosts come back:
> > > 
> > > # corosync-cmapctl | grep 323224576
> > > #
> > > # cibadmin --query|grep 323224576
> > >       <node id="3232245767" uname="atlas6"/>
> > >       <node id="3232245763" uname="atlas2"/>
> > >       <node id="3232245764" uname="atlas3"/>
> > >       <node id="3232245766" uname="atlas5"/>
> > >     <node_state id="3232245767" uname="atlas6" in_ccm="false"
> > > crmd="offline" expected="down" crm-debug-
> > > origin="post_cache_update"/>
> > >     <node_state id="3232245763" uname="atlas2" in_ccm="false"
> > > crmd="offline" expected="down" crm-debug-
> > > origin="post_cache_update"/>
> > >     <node_state id="3232245764" uname="atlas3" in_ccm="false"
> > > crmd="offline" expected="down" crm-debug-
> > > origin="post_cache_update"/>
> > >     <node_state id="3232245766" uname="atlas5" in_ccm="false"
> > > crmd="offline" expected="down" crm-debug-
> > > origin="post_cache_update"/>
> > > 
> > > How could the old node ids be removed completely? crm_node -R does not
> > > help, deleting by cibadmin does not help. What is the proper way to
> > > cleanup the state?
> > > 
> > > Best regards,
> > > Jozsef
> > > --
> > > E-mail : kadlecsik.jozsef at wigner.hun-ren.hu
> > > Address: Wigner Research Centre for Physics
> > >          H-1525 Budapest 114, POB. 49, Hungary
> > > _______________________________________________
> > > Manage your subscription:
> > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > 
> > > ClusterLabs home: https://www.clusterlabs.org/
> > 
> 
> -- 
> E-mail : kadlecsik.jozsef at wigner.hun-ren.hu
> Address: Wigner Research Centre for Physics
>          H-1525 Budapest 114, POB. 49, Hungary

-- 
E-mail : kadlecsik.jozsef at wigner.hun-ren.hu
Address: Wigner Research Centre for Physics
         H-1525 Budapest 114, POB. 49, Hungary