[ClusterLabs] How to delete a node, really, really?

[Kadlecsik József]

Hello,

During an upgrade it turned out that our old node id-s (generated 
many-many years ago) are not accepted anymore. So we renumbered the nodes 
and since then have been unable to delete the old id-s from the system.

Corosync knows the new id-s only and work fine:

# corosync-cmapctl | grep 323224576
# 

Pacemaker knows the new id-s according to the config file and all traces 
are deleted from the cib as well:

# cibadmin --query|grep 323224576
#

Everything seems ok. However, as soon as just corosync stopped or started 
on a standby node, the old ghosts come back:

# corosync-cmapctl | grep 323224576
#
# cibadmin --query|grep 323224576
      <node id="3232245767" uname="atlas6"/>
      <node id="3232245763" uname="atlas2"/>
      <node id="3232245764" uname="atlas3"/>
      <node id="3232245766" uname="atlas5"/>
    <node_state id="3232245767" uname="atlas6" in_ccm="false" 
crmd="offline" expected="down" crm-debug-origin="post_cache_update"/>
    <node_state id="3232245763" uname="atlas2" in_ccm="false" 
crmd="offline" expected="down" crm-debug-origin="post_cache_update"/>
    <node_state id="3232245764" uname="atlas3" in_ccm="false" 
crmd="offline" expected="down" crm-debug-origin="post_cache_update"/>
    <node_state id="3232245766" uname="atlas5" in_ccm="false" 
crmd="offline" expected="down" crm-debug-origin="post_cache_update"/>

How could the old node ids be removed completely? crm_node -R does not 
help, deleting by cibadmin does not help. What is the proper way to 
cleanup the state?

Best regards,
Jozsef
-- 
E-mail : kadlecsik.jozsef at wigner.hun-ren.hu
Address: Wigner Research Centre for Physics
         H-1525 Budapest 114, POB. 49, Hungary