[ClusterLabs] DRBD Insufficient Privileges Error

[Ken Gaillot]

On 11/20/2016 01:58 PM, Jasim Alam wrote:
> Hi,
> 
>  
> 
> I am trying to setup  two node H/A cluster with DRBD. Following is my
> configuration
> 
>  
> 
> /[root at node-1 ~]# pcs config/
> 
> /Cluster Name: Cluster-1/
> 
> /Corosync Nodes:/
> 
> /node-1 node-2 /
> 
> /Pacemaker Nodes:/
> 
> /node-1 node-2 /
> 
> / /
> 
> /Resources: /
> 
> / Resource: vip (class=ocf provider=heartbeat type=IPaddr2)/
> 
> /  Attributes: ip=103.9.185.211 cidr_netmask=32 /
> 
> /  Operations: start interval=0s timeout=20s (vip-start-interval-0s)/
> 
> /              stop interval=0s timeout=20s (vip-stop-interval-0s)/
> 
> /              monitor interval=30s (vip-monitor-interval-30s)/
> 
> /Resource: apache (class=ocf provider=heartbeat type=apache)/
> 
> /  Attributes: configfile=/etc/httpd/conf/httpd.conf
> statusurl=http://localhost/server-status /
> 
> /  Operations: start interval=0s timeout=40s (apache-start-interval-0s)/
> 
> /              stop interval=0s timeout=60s (apache-stop-interval-0s)/
> 
> /              monitor interval=1min (apache-monitor-interval-1min)/
> 
> /Master: StorageClone/
> 
> /  Meta Attrs: master-max=1 master-node-max=1 clone-max=2
> clone-node-max=1 notify=true /
> 
> /  Resource: storage (class=ocf provider=linbit type=drbd)/
> 
> /   Attributes: drbd_resource=drbd0 /
> 
> /   Operations: start interval=0s timeout=240 (storage-start-interval-0s)/
> 
> /               promote interval=0s timeout=90
> (storage-promote-interval-0s)/
> 
> /               demote interval=0s timeout=90 (storage-demote-interval-0s)/
> 
> /               stop interval=0s timeout=100 (storage-stop-interval-0s)/
> 
> /               monitor interval=60s (storage-monitor-interval-60s)/
> 
> / /
> 
> /Stonith Devices: /
> 
> /Fencing Levels: /
> 
> / /
> 
> /Location Constraints:/
> 
> /  Resource: apache/
> 
> /    Enabled on: node-1 (score:50) (id:location-apache-node-1-50)/
> 
> /Ordering Constraints:/
> 
> /  start vip then start apache (kind:Mandatory)
> (id:order-vip-apache-mandatory)/
> 
> /Colocation Constraints:/
> 
> /  vip with apache (score:INFINITY) (id:colocation-vip-apache-INFINITY)/
> 
> / /
> 
> /Resources Defaults:/
> 
> /No defaults set/
> 
> /Operations Defaults:/
> 
> /No defaults set/
> 
> / /
> 
> /Cluster Properties:/
> 
> /cluster-infrastructure: corosync/
> 
> /cluster-name: Cluster-1/
> 
> /dc-version: 1.1.13-10.el7_2.4-44eb2dd/
> 
> /have-watchdog: false/
> 
> /no-quorum-policy: ignore/
> 
> /stonith-enabled: false/
> 
>  
> 
> The problem is I am getting insufficient privilege error on second node
> 
>  
> 
> /[root at node-1 ~]# pcs status/
> 
> /Cluster name: PSD-1/
> 
> /Last updated: Mon Nov 21 01:44:52 2016          Last change: Mon Nov 21
> 01:19:17 2016 by root via cibadmin on node-1/
> 
> /Stack: corosync/
> 
> /Current DC: node-1 (version 1.1.13-10.el7_2.4-44eb2dd) - partition with
> quorum/
> 
> /2 nodes and 4 resources configured/
> 
> / /
> 
> /Online: [ node-1 node-2 ]/
> 
> / /
> 
> /Full list of resources:/
> 
> / /
> 
> /vip    (ocf::heartbeat:IPaddr2):       Started node-1/
> 
> /apache (ocf::heartbeat:apache):        Started node-1/
> 
> /Master/Slave Set: StorageClone [storage]/
> 
> /     storage    (ocf::linbit:drbd):     FAILED node-2 (unmanaged)/
> 
> /     Masters: [ node-1 ]/
> 
> / /
> 
> /Failed Actions:/
> 
> /* storage_stop_0 on node-2 'insufficient privileges' (4): call=16,
> status=complete, exitreason='none',/
> 
> /    last-rc-change='Mon Nov 21 01:19:17 2016', queued=0ms, exec=2ms/
> 
> / /
> 
> / /
> 
> /PCSD Status:/
> 
> /  node-1: Online/
> 
> /  node-2: Online/
> 
> / /
> 
> /Daemon Status:/
> 
> /  corosync: active/disabled/
> 
> /  pacemaker: active/disabled/
> 
> /  pcsd: active/enabled/
> 
>  
> 
> but DRBD seems okay for both nodes
> 
>  
> 
> /  [root at node-1 ~]# drbd-overview /
> 
> / 0:drbd0/0  Connected Primary/Secondary UpToDate/UpToDate/
> 
> //
> 
> / [root at node-2 ~]# drbd-overview /
> 
> / 0:drbd0/0  Connected Secondary/Primary UpToDate/UpToDate/
> 
>  
> 
> Log of node2
> 
>  
> 
> /[root at node-2 ~]# tail -n 10 /var/log/messages/
> 
> /Nov 21 01:19:17 node-2 crmd[4060]:  notice: State transition S_NOT_DC
> -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL
> origin=do_election_count_vote ]/
> 
> /Nov 21 01:19:17 node-2 crmd[4060]:  notice: State transition S_PENDING
> -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE
> origin=do_cl_join_finalize_respond ]/
> 
> /Nov 21 01:19:17 node-2 crmd[4060]:   error: Failed to retrieve
> meta-data for ocf:linbit:drbd/

Pacemaker is unable to get the metadata for the resource agent. Try
getting it manually:

 OCF_ROOT=/usr/lib/ocf /usr/lib/ocf/resource.d/linbit/drbd meta-data

If that works, check whether selinux is blocking it.

> 
> /Nov 21 01:19:17 node-2 crmd[4060]: warning: No metadata found for
> drbd::ocf:linbit: Input/output error (-5)/
> 
> /Nov 21 01:19:17 node-2 crmd[4060]:   error: No metadata for
> linbit::ocf:drbd/
> 
> /Nov 21 01:19:17 node-2 crmd[4060]:  notice: Operation
> storage_monitor_0: insufficient privileges (node=node-2, call=14, rc=4,
> cib-update=17, confirmed=true)/
> 
> /Nov 21 01:19:17 node-2 crmd[4060]:  notice: Operation storage_notify_0:
> ok (node=node-2, call=15, rc=0, cib-update=0, confirmed=true)/
> 
> /Nov 21 01:19:17 node-2 crmd[4060]:  notice: Operation storage_stop_0:
> insufficient privileges (node=node-2, call=16, rc=4, cib-update=18,
> confirmed=true)/
> 
> /Nov 21 01:20:31 node-2 systemd-logind: Removed session 3./
> 
> /Nov 21 01:22:58 node-2 systemd-logind: Removed session 2./
> 
> / /
> 
> Would appreciate any way out of this.
> 
>  
> 
> Thanks,
> 
> Jasim