[ClusterLabs] Unable to communicate with z2-server-nat2 and Unable to synchronize and save tokens on nodes

Akaraphan Janetheerapong jackschariot at gmail.com
Tue Apr 5 05:29:06 EDT 2022


Hi,

I am new to using Corosync and Pacemaker. I have been following this
tutorial:

https://www.ateam-oracle.com/post/isv-implementation-details-part-4a-linux-clustering-with-pacemaker-and-corosync

However, I am stuck in the auth part:

[root at z2-server-nat1 user1]# pcs --debug cluster auth z2-server-nat1
z2-server-nat2 -u hacluster -p xxxxxxxxxx --force
Running: /usr/bin/ruby -I/usr/lib/pcsd/ /usr/lib/pcsd/pcsd-cli.rb auth
Environment:
  GEM_HOME=/usr/lib/pcsd/vendor/bundle/ruby
  HISTSIZE=1000
  HOME=/root
  HOSTNAME=z2-server-nat1
  LANG=en_US.UTF-8
  LC_ALL=C
  LESSOPEN=||/usr/bin/lesspipe.sh %s
  LOGNAME=root

LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
  MAIL=/var/spool/mail/user1
  PATH=/root/bin:/sbin:/bin:/usr/sbin:/usr/bin
  PCSD_DEBUG=true
  PCSD_NETWORK_TIMEOUT=60
  PWD=/home/user1
  SHELL=/bin/bash
  SHLVL=1
  SUDO_COMMAND=/bin/bash
  SUDO_GID=1000
  SUDO_UID=1000
  SUDO_USER=user1
  TERM=xterm
  USER=root
  USERNAME=root
  XDG_SESSION_ID=52
  _=/sbin/pcs
--Debug Input Start--
{"username": "hacluster", "local": false, "nodes": {"z2-server-nat2": null,
"z2-server-nat1": null}, "password": "xxxxxxxxxx", "force": true}
--Debug Input End--

Finished running: /usr/bin/ruby -I/usr/lib/pcsd/ /usr/lib/pcsd/pcsd-cli.rb
auth
Return value: 0
--Debug Stdout Start--
{
  "status": "ok",
  "data": {
    "auth_responses": {
      "z2-server-nat2": {
        "status": "noresponse"
      },
      "z2-server-nat1": {
        "status": "ok",
        "token": "4c1d7eb9-549a-4e53-9c8d-30b735276384"
      }
    },
    "sync_successful": true,
    "sync_nodes_err": [
      "z2-server-nat2"
    ],
    "sync_responses": {
      "z2-server-nat2": {
        "status": "notauthorized"
      },
      "z2-server-nat1": {
        "status": "ok",
        "result": {
          "tokens": "accepted"
        }
      }
    }
  },
  "log": [
    "I, [2022-04-05T09:16:12.482746 #15712]  INFO -- : PCSD Debugging
enabled\n",
    "D, [2022-04-05T09:16:12.483179 #15712] DEBUG -- : Did not detect RHEL
6\n",
    "D, [2022-04-05T09:16:12.483213 #15712] DEBUG -- : Detected systemd is
in use\n",
    "I, [2022-04-05T09:16:12.608604 #15712]  INFO -- : Running:
/usr/sbin/corosync-cmapctl totem.cluster_name\n",
    "I, [2022-04-05T09:16:12.608727 #15712]  INFO -- : CIB USER: hacluster,
groups: \n",
    "D, [2022-04-05T09:16:12.616468 #15712] DEBUG -- :
[\"totem.cluster_name (str) = haproxy-cluseter0\\n\"]\n",
    "D, [2022-04-05T09:16:12.616601 #15712] DEBUG -- : []\n",
    "D, [2022-04-05T09:16:12.616642 #15712] DEBUG -- : Duration:
0.007706732s\n",
    "I, [2022-04-05T09:16:12.616763 #15712]  INFO -- : Return Value: 0\n",
    "I, [2022-04-05T09:16:12.627240 #15712]  INFO -- : No response from:
z2-server-nat2 request: auth, error: couldnt_connect\n",
    "I, [2022-04-05T09:16:12.820609 #15712]  INFO -- : Running:
/usr/sbin/pcs status nodes corosync\n",
    "I, [2022-04-05T09:16:12.820672 #15712]  INFO -- : CIB USER: hacluster,
groups: \n",
    "D, [2022-04-05T09:16:13.029178 #15712] DEBUG -- : [\"Corosync
Nodes:\\n\", \" Online: z2-server-nat1 z2-server-nat2\\n\", \"
Offline:\\n\"]\n",
    "D, [2022-04-05T09:16:13.029316 #15712] DEBUG -- : []\n",
    "D, [2022-04-05T09:16:13.029351 #15712] DEBUG -- : Duration:
0.20848278s\n",
    "I, [2022-04-05T09:16:13.029514 #15712]  INFO -- : Return Value: 0\n",
    "I, [2022-04-05T09:16:13.029855 #15712]  INFO -- : Sending config
'tokens' version 31 505875ea7aca471f78f3de3d9d622c7996c1961a to nodes:
z2-server-nat1, z2-server-nat2\n",
    "I, [2022-04-05T09:16:13.030258 #15712]  INFO -- : SRWT Node:
z2-server-nat1 Request: set_configs\n",
    "I, [2022-04-05T09:16:13.030884 #15712]  INFO -- : SRWT Node:
z2-server-nat2 Request: set_configs\n",
    "E, [2022-04-05T09:16:13.030945 #15712] ERROR -- : Unable to connect to
node z2-server-nat2, no token available\n",
    "I, [2022-04-05T09:16:13.055544 #15712]  INFO -- : Sending config
response from z2-server-nat2: {\"status\"=>\"notauthorized\"}\n",
    "I, [2022-04-05T09:16:13.055614 #15712]  INFO -- : Sending config
response from z2-server-nat1: {\"status\"=>\"ok\",
\"result\"=>{\"tokens\"=>\"accepted\"}}\n"
  ]
}

--Debug Stdout End--
--Debug Stderr Start--

--Debug Stderr End--

Error: Unable to communicate with z2-server-nat2
z2-server-nat1: Authorized
Error: Unable to synchronize and save tokens on nodes: z2-server-nat2. Are
they authorized?


------------------------------------------------------------------------------------------------------------------------

I've also applied the iptables:

[root at z2-server-nat1 user1]# sudo iptables -S
......
-A IN_public_allow -p udp -m udp --dport 5404 -m conntrack --ctstate
NEW,UNTRACKED -j ACCEPT
-A IN_public_allow -p udp -m udp --dport 5405 -m conntrack --ctstate
NEW,UNTRACKED -j ACCEPT
-A IN_public_allow -p udp -m udp --dport 5406 -m conntrack --ctstate
NEW,UNTRACKED -j ACCEPT

------------------------------------------------------------------------------------------------------------------------

I've also made sure that the nodes see each other:

[root at z2-server-nat1 user1]# ping z2-server-nat2
PING z2-server-nat2 (10.0.0.3) 56(84) bytes of data.
64 bytes from z2-server-nat2 (10.0.0.3): icmp_seq=1 ttl=64 time=0.427 ms
64 bytes from z2-server-nat2 (10.0.0.3): icmp_seq=2 ttl=64 time=0.387 ms
64 bytes from z2-server-nat2 (10.0.0.3): icmp_seq=3 ttl=64 time=0.398 ms

------------------------------------------------------------------------------------------------------------------------

I've also made sure that hostnames are in /etc/hosts of both server:

[root at z2-server-nat1 user1]# cat /etc/hosts
...
...
x.x.x.3 z2-server-nat2
x.x.x.2 z2-server-nat1
...
...

------------------------------------------------------------------------------------------------------------------------

I've also made sure the service is up:

[user1 at z2-server-nat2 ~]$ systemctl status pcsd.service
● pcsd.service - PCS GUI and remote configuration interface
   Loaded: loaded (/usr/lib/systemd/system/pcsd.service; enabled; vendor
preset: disabled)
   Active: active (running) since Tue 2022-04-05 04:29:16 GMT; 3h 24min ago
     Docs: man:pcsd(8)
           man:pcs(8)
 Main PID: 856 (pcsd)
   Memory: 28.6M
   CGroup: /system.slice/pcsd.service
           └─856 /usr/bin/ruby /usr/lib/pcsd/pcsd

Apr 05 04:29:16 z2-server-nat2 systemd[1]: Starting PCS GUI and remote
configuration interface...
Apr 05 04:29:16 z2-server-nat2 systemd[1]: Started PCS GUI and remote
configuration interface.

------------------------------------------------------------------------------------------------------------------------

Am I missing something in making the nodes able to communicate with each
other? How do I proceed from here?

Regards,
Chariot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20220405/bf728ca5/attachment.htm>


More information about the Users mailing list