[ClusterLabs] Pacemaker newbie needs advice for adding a new cluster node

thl at it-hluchnik.de thl at it-hluchnik.de
Thu Aug 11 11:20:24 UTC 2016 

Hello all,

maybe someone can help me adding a node in my test cluster.

I have a working three-node test Cluster in my VirtualBox, using OEL7.2 and now I try to add another node but I have no plan and I did some kind of try & error which does not work.

This is my three-node cluster:

# pcs status
Cluster name: zweiter
Last updated: Thu Aug 11 12:53:55 2016          Last change: Thu Aug 11 12:53:47 2016 by root via cibadmin on knoten01-hb
Stack: corosync
Current DC: knoten01-hb (version 1.1.13-10.el7-44eb2dd) - partition with quorum
3 nodes and 12 resources configured

Online: [ knoten01-hb knoten02-hb knoten03-hb ]

Full list of resources:

 Clone Set: ResO2CB-clone [ResO2CB]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]
 Clone Set: ResOCFS2_00-clone [ResOCFS2_00]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]
 Clone Set: ClusterIP-clone [ClusterIP] (unique)
     ClusterIP:0        (ocf::heartbeat:IPaddr2):       Started knoten01-hb
     ClusterIP:1        (ocf::heartbeat:IPaddr2):       Started knoten02-hb
     ClusterIP:2        (ocf::heartbeat:IPaddr2):       Started knoten03-hb
 Clone Set: WebSite-clone [WebSite]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]

PCSD Status:
  knoten01-hb: Online
  knoten02-hb: Online
  knoten03-hb: Online

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled




I setup another host with name knoten04-hb, pcsd is started by systemd. My first idea was modifying /etc/corosync/corosync.conf on all four hosts, adding the new host. When done, without any stop/start action, pcs shows me this:

# pcs status
Cluster name: zweiter
Last updated: Thu Aug 11 13:02:22 2016          Last change: Thu Aug 11 12:53:47 2016 by root via cibadmin on knoten01-hb
Stack: corosync
Current DC: knoten01-hb (version 1.1.13-10.el7-44eb2dd) - partition with quorum
3 nodes and 12 resources configured

Online: [ knoten01-hb knoten02-hb knoten03-hb ]

Full list of resources:

 Clone Set: ResO2CB-clone [ResO2CB]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]
 Clone Set: ResOCFS2_00-clone [ResOCFS2_00]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]
 Clone Set: ClusterIP-clone [ClusterIP] (unique)
     ClusterIP:0        (ocf::heartbeat:IPaddr2):       Started knoten01-hb
     ClusterIP:1        (ocf::heartbeat:IPaddr2):       Started knoten02-hb
     ClusterIP:2        (ocf::heartbeat:IPaddr2):       Started knoten03-hb
 Clone Set: WebSite-clone [WebSite]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]

PCSD Status:
  knoten01-hb: Online
  knoten02-hb: Online
  knoten03-hb: Online
  knoten04-hb: Online

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled


PCSD Status shows the new node, everything else is unchanged. I guess the corosync part is OK.




Next step is getting pacemaker configured. I tried:

# cibadmin --query > add_knoten04-hb_cfg
# vi add_knoten04-hb_cfg
# cibadmin --replace --xml-file add_knoten04-hb_cfg

The only entry I changed was this:

    <nodes>
      <node id="1" uname="knoten01-hb"/>
      <node id="2" uname="knoten02-hb"/>
      <node id="3" uname="knoten03-hb"/>
      <node id="4" uname="knoten04-hb"/>        <==============
    </nodes>




Again, this looks good:

# pcs status
Cluster name: zweiter
Last updated: Thu Aug 11 13:08:19 2016          Last change: Thu Aug 11 13:08:11 2016 by root via cibadmin on knoten01-hb
Stack: corosync
Current DC: knoten01-hb (version 1.1.13-10.el7-44eb2dd) - partition with quorum
4 nodes and 13 resources configured

Online: [ knoten01-hb knoten02-hb knoten03-hb ]
OFFLINE: [ knoten04-hb ]

Full list of resources:

 Clone Set: ResO2CB-clone [ResO2CB]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]
 Clone Set: ResOCFS2_00-clone [ResOCFS2_00]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]
 Clone Set: ClusterIP-clone [ClusterIP] (unique)
     ClusterIP:0        (ocf::heartbeat:IPaddr2):       Started knoten01-hb
     ClusterIP:1        (ocf::heartbeat:IPaddr2):       Started knoten02-hb
     ClusterIP:2        (ocf::heartbeat:IPaddr2):       Started knoten03-hb
 Clone Set: WebSite-clone [WebSite]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]
     Stopped: [ knoten04-hb ]

PCSD Status:
  knoten01-hb: Online
  knoten02-hb: Online
  knoten03-hb: Online
  knoten04-hb: Online

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled


knoten04-hb is known but offline, for sure. OK, let's start it:




# pcs cluster start knoten04-hb
knoten04-hb: Starting Cluster...

The funny thing: pcs status on the old nodes tell me:

[root at knoten01 ~]# pcs status
Cluster name: zweiter
Last updated: Thu Aug 11 13:10:45 2016          Last change: Thu Aug 11 13:08:11 2016 by root via cibadmin on knoten01-hb
Stack: corosync
Current DC: knoten01-hb (version 1.1.13-10.el7-44eb2dd) - partition with quorum
4 nodes and 13 resources configured

Online: [ knoten01-hb knoten02-hb knoten03-hb ]
OFFLINE: [ knoten04-hb ]

Full list of resources:

 Clone Set: ResO2CB-clone [ResO2CB]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]
 Clone Set: ResOCFS2_00-clone [ResOCFS2_00]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]
 Clone Set: ClusterIP-clone [ClusterIP] (unique)
     ClusterIP:0        (ocf::heartbeat:IPaddr2):       Started knoten01-hb
     ClusterIP:1        (ocf::heartbeat:IPaddr2):       Started knoten02-hb
     ClusterIP:2        (ocf::heartbeat:IPaddr2):       Started knoten03-hb
 Clone Set: WebSite-clone [WebSite]
     Started: [ knoten01-hb knoten02-hb knoten03-hb ]
     Stopped: [ knoten04-hb ]

PCSD Status:
  knoten01-hb: Online
  knoten02-hb: Online
  knoten03-hb: Online
  knoten04-hb: Online

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled


And pcs on the new node tells me that:

[root at knoten04 ~]# pcs status
Cluster name: zweiter
Last updated: Thu Aug 11 13:10:25 2016          Last change: Thu Aug 11 12:40:38 2016 by hacluster via crmd on knoten04-hb
Stack: corosync
Current DC: knoten04-hb (version 1.1.13-10.el7-44eb2dd) - partition WITHOUT quorum
4 nodes and 13 resources configured

Online: [ knoten04-hb ]
OFFLINE: [ knoten01-hb knoten02-hb knoten03-hb ]

Full list of resources:

 Clone Set: ResO2CB-clone [ResO2CB]
     Stopped: [ knoten01-hb knoten02-hb knoten03-hb knoten04-hb ]
 Clone Set: ResOCFS2_00-clone [ResOCFS2_00]
     Stopped: [ knoten01-hb knoten02-hb knoten03-hb knoten04-hb ]
 Clone Set: ClusterIP-clone [ClusterIP] (unique)
     ClusterIP:0        (ocf::heartbeat:IPaddr2):       Stopped
     ClusterIP:1        (ocf::heartbeat:IPaddr2):       Stopped
     ClusterIP:2        (ocf::heartbeat:IPaddr2):       Stopped
 Clone Set: WebSite-clone [WebSite]
     Stopped: [ knoten01-hb knoten02-hb knoten03-hb knoten04-hb ]

PCSD Status:
  knoten01-hb: Online
  knoten02-hb: Online
  knoten03-hb: Online
  knoten04-hb: Online

Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled


This is obviously no valid cluster. So what am I doing wrong? How to add the node, getting a working four-node cluster?

Thanks in advance for any help.

Thomas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part.
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20160811/150ddcee/attachment-0003.sig>

More information about the Users mailing list