[ClusterLabs] Node add doesn't add node?

[Israel Brewster]

So in my ongoing work to upgrade my cluster to CentOS 7, I got one box up and running on CentOS 7, with the cluster fully configured and functional, and moved all my services over to it. Now I'm trying to add a second node, following the directions here:

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/high_availability_add-on_reference/s1-clusternodemanage-haar#s2-nodeadd-HAAR

However, it doesn't appear to be working. The existing node is named "follow3", and the new node I am trying to add is named "follow1":

- The auth command run from follow3 returns "follow1: Authorized", so that looks good.
- The "pcs cluster node add follow1" command, again run on follow3, gives the following output:

Disabling SBD service...
follow1: sbd disabled
Sending remote node configuration files to 'follow1'
follow1: successful distribution of the file 'pacemaker_remote authkey'
follow3: Corosync updated
Setting up corosync...
follow1: Succeeded
Synchronizing pcsd certificates on nodes follow1...
follow1: Success
Restarting pcsd on the nodes in order to reload the certificates...
follow1: Success

...So it would appear that that worked as well. I then issued the "pcs cluster start --all" command, which gave the following output:

[root at follow3 ~]# pcs cluster start --all
follow3: Starting Cluster (corosync)...
follow1: Starting Cluster (corosync)...
follow3: Starting Cluster (pacemaker)...
follow1: Starting Cluster (pacemaker)...

So again, everything looks good (to me). However, when I run "pcs status" on the existing node, I get the following:

[root at follow3 ~]# pcs status
Cluster name: follow
Stack: corosync
Current DC: follow3 (version 1.1.19-8.el7_6.2-c3c624ea3d) - partition with quorum
Last updated: Thu Jan 10 10:47:33 2019
Last change: Wed Jan  9 21:39:37 2019 by root via cibadmin on follow3

1 node configured
29 resources configured

Online: [ follow3 ]

Full list of resources:

which would seem to indicate that it doesn't know about the node I just added (follow1). Meanwhile, follow1 "pcs status" shows this:

[root at follow1 ~]# pcs status
Cluster name: follow
Stack: corosync
Current DC: follow1 (version 1.1.19-8.el7_6.2-c3c624ea3d) - partition WITHOUT quorum
Last updated: Thu Jan 10 10:54:25 2019
Last change: Thu Jan 10 10:54:13 2019 by root via cibadmin on follow1

2 nodes configured
0 resources configured

Online: [ follow1 ]
OFFLINE: [ follow3 ]

No resources


Daemon Status:
  corosync: active/disabled
  pacemaker: active/disabled
  pcsd: active/enabled

So it got at least *some* of the config, but apparently not the full thing (no resources), and it shows follow3 as offline, even though it is online and reachable. Oddly "pcs cluster status" shows both follow1 and follow3 pcsd status as online. What am I missing here?
-----------------------------------------------
Israel Brewster
Systems Analyst II
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
-----------------------------------------------

[cid:da9e47ee-066d-4dcf-888a-fd9cd23813c9 at flyravn.com]



[cid:f03eb32f-5ff5-49b8-926e-5120f8aef3a2 at flyravn.com]







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20190110/4cfe6e19/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2298 bytes
Desc: image001.jpg
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20190110/4cfe6e19/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 6246 bytes
Desc: image002.jpg
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20190110/4cfe6e19/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Israel Brewster.vcf
Type: text/directory
Size: 417 bytes
Desc: Israel Brewster.vcf
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20190110/4cfe6e19/attachment-0001.bin>