[ClusterLabs] PCS cluster auth fails
Tomas Jelinek
tojeline at redhat.com
Fri Feb 7 11:31:26 EST 2020
Hi,
I haven't seen such a traceback before. It looks like there are pcsd
files missing
(/usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/multipart/parser.rb).
Is this the case? If so, can you try reinstalling pcs packages on your
nodes?
Regards,
Tomas
Dne 07. 02. 20 v 7:39 Somanath Jeeva napsal(a):
> Hi ,
>
> I am using a two node corosync cluster(node1 and node2). When I do pcs
> cluster auth during corosync-qdevice configuration to the qdevice
> node(qnode), I am getting the below error,
>
> /$ sudo pcs cluster auth qnode -u hacluster -p **************** --debug/
>
> /Running: /usr/bin/ruby -I/usr/lib/pcsd/ /usr/lib/pcsd/pcsd-cli.rb auth/
>
> /Environment:/
>
> / GEM_HOME=/usr/lib/pcsd/vendor/bundle/ruby/
>
> / HISTSIZE=1000/
>
> / HOME=/root/
>
> / HOSTNAME=node1/
>
> / LANG=en_US.UTF-8/
>
> / LC_ALL=C/
>
> / LOGNAME=root/
>
> /
> LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:/
>
> / MAIL=/var/spool/mail/admin/
>
> / PATH=/sbin:/bin:/usr/sbin:/usr/bin/
>
> / PCSD_DEBUG=true/
>
> / PCSD_NETWORK_TIMEOUT=60/
>
> / PS1=[\u@\h-$node_name \W]\$ /
>
> / SHELL=/sbin/nologin/
>
> / SUDO_COMMAND=/sbin/pcs cluster auth qnode -u hacluster -p ***********
> --debug/
>
> / SUDO_GID=5007/
>
> / SUDO_UID=5008/
>
> / SUDO_USER=admin/
>
> / TERM=xterm/
>
> / USER=root/
>
> / USERNAME=root/
>
> /--Debug Input Start--/
>
> /{"username": "hacluster", "local": false, "nodes": {"qnode": null},
> "password": "***********", "force": false}/
>
> /--Debug Input End--/
>
> //
>
> /Finished running: /usr/bin/ruby -I/usr/lib/pcsd/
> /usr/lib/pcsd/pcsd-cli.rb auth/
>
> /Return value: 0/
>
> /--Debug Stdout Start--/
>
> /{/
>
> / "status": "ok",/
>
> / "data": {/
>
> / "auth_responses": {/
>
> / "qnode": {/
>
> / "status": "ok",/
>
> / "token": "66c1020d-1089-4d8b-beab-a8f76e2c8b89"/
>
> / }/
>
> / },/
>
> / "sync_successful": true,/
>
> / "sync_nodes_err": [/
>
> / "node2"/
>
> / ],/
>
> / "sync_responses": {/
>
> / "node2": {/
>
> / "status": "error"/
>
> / },/
>
> / "node1": {/
>
> / "status": "ok",/
>
> / "result": {/
>
> / "tokens": "accepted"/
>
> / }/
>
> / }/
>
> / }/
>
> / },/
>
> / "log": [/
>
> / "I, [2020-02-07T17:11:34.011890 #30323] INFO -- : PCSD Debugging
> enabled\n",/
>
> / "D, [2020-02-07T17:11:34.012401 #30323] DEBUG -- : Did not detect
> RHEL 6\n",/
>
> / "D, [2020-02-07T17:11:34.012446 #30323] DEBUG -- : Detected systemd
> is in use\n",/
>
> / "I, [2020-02-07T17:11:34.160540 #30323] INFO -- : Running:
> /usr/sbin/corosync-cmapctl totem.cluster_name\n",/
>
> / "I, [2020-02-07T17:11:34.160707 #30323] INFO -- : CIB USER:
> hacluster, groups: \n",/
>
> / "D, [2020-02-07T17:11:34.176123 #30323] DEBUG -- :
> [\"totem.cluster_name (str) = HBASE\\n\"]\n",/
>
> / "D, [2020-02-07T17:11:34.176307 #30323] DEBUG -- : []\n",/
>
> / "D, [2020-02-07T17:11:34.176357 #30323] DEBUG -- : Duration:
> 0.015370265s\n",/
>
> / "I, [2020-02-07T17:11:34.176443 #30323] INFO -- : Return Value: 0\n",/
>
> / "I, [2020-02-07T17:11:34.407886 #30323] INFO -- : Running:
> /usr/sbin/pcs status nodes corosync\n",/
>
> / "I, [2020-02-07T17:11:34.407989 #30323] INFO -- : CIB USER:
> hacluster, groups: \n",/
>
> / "D, [2020-02-07T17:11:34.775751 #30323] DEBUG -- : [\"Corosync
> Nodes:\\n\", \" Online: node2 node1\\n\", \" Offline:\\n\"]\n",/
>
> / "D, [2020-02-07T17:11:34.775909 #30323] DEBUG -- : []\n",/
>
> / "D, [2020-02-07T17:11:34.775960 #30323] DEBUG -- : Duration:
> 0.367740571s\n",/
>
> / "I, [2020-02-07T17:11:34.776056 #30323] INFO -- : Return Value: 0\n",/
>
> / "I, [2020-02-07T17:11:34.776517 #30323] INFO -- : Sending config
> 'tokens' version 5 40712570bb7e5718afce943a151b259f04e7c080 to nodes:
> node2, node1\n",/
>
> / "I, [2020-02-07T17:11:34.777079 #30323] INFO -- : SRWT Node: node2
> Request: set_configs\n",/
>
> / "I, [2020-02-07T17:11:34.777695 #30323] INFO -- : SRWT Node: node1
> Request: set_configs\n",/
>
> / "I, [2020-02-07T17:11:34.865021 #30323] INFO -- : Sending config
> response from node2: {\"status\"=>\"error\"}\n",/
>
> / "I, [2020-02-07T17:11:34.865141 #30323] INFO -- : Sending config
> response from node1: {\"status\"=>\"ok\",
> \"result\"=>{\"tokens\"=>\"accepted\"}}\n"/
>
> / ]/
>
> /}/
>
> //
>
> /--Debug Stdout End--/
>
> /--Debug Stderr Start--/
>
> //
>
> /--Debug Stderr End--/
>
> //
>
> /qnode: Authorized/
>
> /Error: Unable to synchronize and save tokens on nodes: node2. Are they
> authorized?/
>
> Initially I got synchronize error for node1 and node2. After pcsd
> restart issue got resolved in node1 and now issue occurs only in node2.
> Even in debug logs I am not able to get the exact error message. During
> the auth , I am getting the below error in pcsd log,
>
> /2020-02-07 17:11:34] ERROR LoadError: cannot load such file --
> rack/multipart/parser/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/multipart.rb:25:in
> `parse_multipart'/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/request.rb:375:in
> `parse_multipart'/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/request.rb:207:in
> `POST'/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/methodoverride.rb:43:in
> `method_override_param'/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/methodoverride.rb:27:in
> `method_override'/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/methodoverride.rb:15:in
> `call'/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/sinatra-1.4.8/lib/sinatra/base.rb:182:in
> `call'/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/sinatra-1.4.8/lib/sinatra/base.rb:2013:in
> `call'/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in
> `block in call'/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/sinatra-1.4.8/lib/sinatra/base.rb:1787:in
> `synchronize'/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in
> `call'/
>
> /
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/handler/webrick.rb:88:in
> `service'/
>
> / /usr/share/ruby/webrick/httpserver.rb:138:in `service'/
>
> / /usr/share/ruby/webrick/httpserver.rb:94:in `run'/
>
> / /usr/share/ruby/webrick/server.rb:295:in `block in
> start_thread'/
>
> Is there any way to know what the error is here.
>
> With Regards
>
> Somanath Thilak J
>
>
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
>
> ClusterLabs home: https://www.clusterlabs.org/
>
More information about the Users
mailing list