[ClusterLabs] PCS cluster auth fails

Tomas Jelinek tojeline at redhat.com
Fri Feb 7 11:31:26 EST 2020


Hi,

I haven't seen such a traceback before. It looks like there are pcsd 
files missing 
(/usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/multipart/parser.rb). 
Is this the case? If so, can you try reinstalling pcs packages on your 
nodes?

Regards,
Tomas


Dne 07. 02. 20 v 7:39 Somanath Jeeva napsal(a):
> Hi ,
> 
> I am using a two node corosync cluster(node1 and node2). When I do pcs 
> cluster auth during corosync-qdevice configuration to the qdevice 
> node(qnode), I am getting the below error,
> 
> /$ sudo pcs cluster auth qnode -u hacluster -p **************** --debug/
> 
> /Running: /usr/bin/ruby -I/usr/lib/pcsd/ /usr/lib/pcsd/pcsd-cli.rb auth/
> 
> /Environment:/
> 
> /  GEM_HOME=/usr/lib/pcsd/vendor/bundle/ruby/
> 
> /  HISTSIZE=1000/
> 
> /  HOME=/root/
> 
> /  HOSTNAME=node1/
> 
> /  LANG=en_US.UTF-8/
> 
> /  LC_ALL=C/
> 
> /  LOGNAME=root/
> 
> /  
> LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:/
> 
> /  MAIL=/var/spool/mail/admin/
> 
> /  PATH=/sbin:/bin:/usr/sbin:/usr/bin/
> 
> /  PCSD_DEBUG=true/
> 
> /  PCSD_NETWORK_TIMEOUT=60/
> 
> /  PS1=[\u@\h-$node_name \W]\$ /
> 
> /  SHELL=/sbin/nologin/
> 
> /  SUDO_COMMAND=/sbin/pcs cluster auth qnode -u hacluster -p *********** 
> --debug/
> 
> /  SUDO_GID=5007/
> 
> /  SUDO_UID=5008/
> 
> /  SUDO_USER=admin/
> 
> /  TERM=xterm/
> 
> /  USER=root/
> 
> /  USERNAME=root/
> 
> /--Debug Input Start--/
> 
> /{"username": "hacluster", "local": false, "nodes": {"qnode": null}, 
> "password": "***********", "force": false}/
> 
> /--Debug Input End--/
> 
> //
> 
> /Finished running: /usr/bin/ruby -I/usr/lib/pcsd/ 
> /usr/lib/pcsd/pcsd-cli.rb auth/
> 
> /Return value: 0/
> 
> /--Debug Stdout Start--/
> 
> /{/
> 
> /  "status": "ok",/
> 
> /  "data": {/
> 
> /    "auth_responses": {/
> 
> /      "qnode": {/
> 
> /        "status": "ok",/
> 
> /        "token": "66c1020d-1089-4d8b-beab-a8f76e2c8b89"/
> 
> /      }/
> 
> /    },/
> 
> /    "sync_successful": true,/
> 
> /    "sync_nodes_err": [/
> 
> /      "node2"/
> 
> /    ],/
> 
> /    "sync_responses": {/
> 
> /      "node2": {/
> 
> /        "status": "error"/
> 
> /      },/
> 
> /      "node1": {/
> 
> /        "status": "ok",/
> 
> /        "result": {/
> 
> /          "tokens": "accepted"/
> 
> /        }/
> 
> /      }/
> 
> /    }/
> 
> /  },/
> 
> /  "log": [/
> 
> /    "I, [2020-02-07T17:11:34.011890 #30323]  INFO -- : PCSD Debugging 
> enabled\n",/
> 
> /    "D, [2020-02-07T17:11:34.012401 #30323] DEBUG -- : Did not detect 
> RHEL 6\n",/
> 
> /    "D, [2020-02-07T17:11:34.012446 #30323] DEBUG -- : Detected systemd 
> is in use\n",/
> 
> /    "I, [2020-02-07T17:11:34.160540 #30323]  INFO -- : Running: 
> /usr/sbin/corosync-cmapctl totem.cluster_name\n",/
> 
> /    "I, [2020-02-07T17:11:34.160707 #30323]  INFO -- : CIB USER: 
> hacluster, groups: \n",/
> 
> /    "D, [2020-02-07T17:11:34.176123 #30323] DEBUG -- : 
> [\"totem.cluster_name (str) = HBASE\\n\"]\n",/
> 
> /    "D, [2020-02-07T17:11:34.176307 #30323] DEBUG -- : []\n",/
> 
> /    "D, [2020-02-07T17:11:34.176357 #30323] DEBUG -- : Duration: 
> 0.015370265s\n",/
> 
> /    "I, [2020-02-07T17:11:34.176443 #30323]  INFO -- : Return Value: 0\n",/
> 
> /    "I, [2020-02-07T17:11:34.407886 #30323]  INFO -- : Running: 
> /usr/sbin/pcs status nodes corosync\n",/
> 
> /    "I, [2020-02-07T17:11:34.407989 #30323]  INFO -- : CIB USER: 
> hacluster, groups: \n",/
> 
> /    "D, [2020-02-07T17:11:34.775751 #30323] DEBUG -- : [\"Corosync 
> Nodes:\\n\", \" Online: node2 node1\\n\", \" Offline:\\n\"]\n",/
> 
> /    "D, [2020-02-07T17:11:34.775909 #30323] DEBUG -- : []\n",/
> 
> /    "D, [2020-02-07T17:11:34.775960 #30323] DEBUG -- : Duration: 
> 0.367740571s\n",/
> 
> /    "I, [2020-02-07T17:11:34.776056 #30323]  INFO -- : Return Value: 0\n",/
> 
> /    "I, [2020-02-07T17:11:34.776517 #30323]  INFO -- : Sending config 
> 'tokens' version 5 40712570bb7e5718afce943a151b259f04e7c080 to nodes: 
> node2, node1\n",/
> 
> /    "I, [2020-02-07T17:11:34.777079 #30323]  INFO -- : SRWT Node: node2 
> Request: set_configs\n",/
> 
> /    "I, [2020-02-07T17:11:34.777695 #30323]  INFO -- : SRWT Node: node1 
> Request: set_configs\n",/
> 
> /    "I, [2020-02-07T17:11:34.865021 #30323]  INFO -- : Sending config 
> response from node2: {\"status\"=>\"error\"}\n",/
> 
> /    "I, [2020-02-07T17:11:34.865141 #30323]  INFO -- : Sending config 
> response from node1: {\"status\"=>\"ok\", 
> \"result\"=>{\"tokens\"=>\"accepted\"}}\n"/
> 
> /  ]/
> 
> /}/
> 
> //
> 
> /--Debug Stdout End--/
> 
> /--Debug Stderr Start--/
> 
> //
> 
> /--Debug Stderr End--/
> 
> //
> 
> /qnode: Authorized/
> 
> /Error: Unable to synchronize and save tokens on nodes: node2. Are they 
> authorized?/
> 
> Initially I got synchronize error for node1 and node2. After pcsd 
> restart issue got resolved in node1 and now issue occurs only in node2.  
> Even in debug logs I am not able to get the exact error message. During 
> the auth , I am getting the below error in pcsd log,
> 
> /2020-02-07 17:11:34] ERROR LoadError: cannot load such file -- 
> rack/multipart/parser/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/multipart.rb:25:in 
> `parse_multipart'/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/request.rb:375:in 
> `parse_multipart'/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/request.rb:207:in 
> `POST'/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/methodoverride.rb:43:in 
> `method_override_param'/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/methodoverride.rb:27:in 
> `method_override'/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/methodoverride.rb:15:in 
> `call'/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/sinatra-1.4.8/lib/sinatra/base.rb:182:in 
> `call'/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/sinatra-1.4.8/lib/sinatra/base.rb:2013:in 
> `call'/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in 
> `block in call'/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/sinatra-1.4.8/lib/sinatra/base.rb:1787:in 
> `synchronize'/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/sinatra-1.4.8/lib/sinatra/base.rb:1487:in 
> `call'/
> 
> /              
> /usr/lib/pcsd/vendor/bundle/ruby/gems/rack-1.6.10/lib/rack/handler/webrick.rb:88:in 
> `service'/
> 
> /              /usr/share/ruby/webrick/httpserver.rb:138:in `service'/
> 
> /              /usr/share/ruby/webrick/httpserver.rb:94:in `run'/
> 
> /              /usr/share/ruby/webrick/server.rb:295:in `block in 
> start_thread'/
> 
> Is there any way to know what the error is here.
> 
> With Regards
> 
> Somanath Thilak J
> 
> 
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
> 
> ClusterLabs home: https://www.clusterlabs.org/
> 


More information about the Users mailing list