[ClusterLabs] NFS4 share not working

solarflow99 solarflow99 at gmail.com
Fri Feb 22 18:57:59 EST 2019 

I'm trying to have my NFS share exported via pacemaker and now it doesn't
seem to be working, it also kills off nfs-mountd.  It looks like the rbd
device could have something to do with it, the nfsroot doesn't get
exported, but there's no indication why:



Feb 22 15:36:23 cephmgr101.corp.mydomain.com crmd[32600]:   notice: The
local CRM is operational
Feb 22 15:36:23 cephmgr101.corp.mydomain.com crmd[32600]:   notice: State
transition S_STARTING -> S_PENDING
Feb 22 15:36:24 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Could
not obtain a node name for corosync nodeid 3
Feb 22 15:36:24 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Could
not obtain a node name for corosync nodeid 2
Feb 22 15:36:24 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Could
not obtain a node name for corosync nodeid 3
Feb 22 15:36:44 cephmgr101.corp.mydomain.com crmd[32600]:   notice: State
transition S_PENDING -> S_NOT_DC
Feb 22 15:36:45 cephmgr101.corp.mydomain.com
nfsserver(p_nfs_server)[32648]: INFO: Status: rpcbind
Feb 22 15:36:45 cephmgr101.corp.mydomain.com
nfsserver(p_nfs_server)[32655]: INFO: Status: nfs-mountd
Feb 22 15:36:45 cephmgr101.corp.mydomain.com
nfsserver(p_nfs_server)[32662]: INFO: Status: nfs-idmapd
Feb 22 15:36:45 cephmgr101.corp.mydomain.com
nfsserver(p_nfs_server)[32674]: INFO: Status: rpc-statd
Feb 22 15:36:45 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Result
of probe operation for p_nfs_server on cephmgr101.corp.mydomain.com: 0 (ok)
Feb 22 15:36:50 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Result
of probe operation for p_rbd_map_1 on cephmgr101.corp.mydomain.com: 7 (not
running)
Feb 22 15:36:50 cephmgr101.corp.mydomain.com
nfsserver(p_nfs_server)[32720]: INFO: Stopping NFS server ...
Feb 22 15:36:50 cephmgr101.corp.mydomain.com systemd[1]: Stopping NFS
server and services...
Feb 22 15:36:50 cephmgr101.corp.mydomain.com systemd[1]: Stopped NFS server
and services.
Feb 22 15:36:50 cephmgr101.corp.mydomain.com systemd[1]: Stopping NFSv4
ID-name mapping service...
Feb 22 15:36:50 cephmgr101.corp.mydomain.com rpc.mountd[32525]: Caught
signal 15, un-registering and exiting.
Feb 22 15:36:50 cephmgr101.corp.mydomain.com systemd[1]: Stopping NFS Mount
Daemon...
Feb 22 15:36:50 cephmgr101.corp.mydomain.com systemd[1]: Stopped NFSv4
ID-name mapping service.
Feb 22 15:36:50 cephmgr101.corp.mydomain.com systemd[1]: Stopped NFS Mount
Daemon.
Feb 22 15:36:50 cephmgr101.corp.mydomain.com kernel: nfsd: last server has
exited, flushing export cache
Feb 22 15:36:50 cephmgr101.corp.mydomain.com
nfsserver(p_nfs_server)[32744]: INFO: Stop: threads
Feb 22 15:36:50 cephmgr101.corp.mydomain.com systemd[1]: Stopping NFS
status monitor for NFSv2/3 locking....
Feb 22 15:36:50 cephmgr101.corp.mydomain.com systemd[1]: Stopped NFS status
monitor for NFSv2/3 locking..
Feb 22 15:36:50 cephmgr101.corp.mydomain.com
nfsserver(p_nfs_server)[32755]: INFO: Stop: rpc-statd
Feb 22 15:36:50 cephmgr101.corp.mydomain.com
nfsserver(p_nfs_server)[32765]: INFO: Stop: nfs-idmapd
Feb 22 15:36:51 cephmgr101.corp.mydomain.com nfsserver(p_nfs_server)[313]:
INFO: Stop: nfs-mountd
Feb 22 15:36:51 cephmgr101.corp.mydomain.com nfsserver(p_nfs_server)[324]:
INFO: Stop: rpc-gssd
Feb 22 15:36:51 cephmgr101.corp.mydomain.com nfsserver(p_nfs_server)[333]:
INFO: Stop: umount (1/10 attempts)
Feb 22 15:36:52 cephmgr101.corp.mydomain.com systemd[1]: Stopped target
rpc_pipefs.target.
Feb 22 15:36:52 cephmgr101.corp.mydomain.com systemd[1]: Stopping
rpc_pipefs.target.
Feb 22 15:36:52 cephmgr101.corp.mydomain.com nfsserver(p_nfs_server)[349]:
INFO: NFS server stopped
Feb 22 15:36:52 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Result
of stop operation for p_nfs_server on cephmgr101.corp.mydomain.com: 0 (ok)
Feb 22 15:36:52 cephmgr101.corp.mydomain.com
exportfs(p_nfs_export_root_1)[411]: INFO: Directory /mnt/nfsroot is not
exported to 172.20.3.0/255.255.255.0 (stopped).
Feb 22 15:36:52 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Result
of probe operation for p_nfs_export_root_1 on cephmgr101.corp.mydomain.com:
7 (not running)
Feb 22 15:36:52 cephmgr101.corp.mydomain.com Filesystem(p_fs_rbd_1)[431]:
WARNING: Couldn't find device [/dev/rbd0]. Expected /dev/??? to exist
Feb 22 15:36:52 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Result
of probe operation for p_fs_rbd_1 on cephmgr101.corp.mydomain.com: 7 (not
running)
Feb 22 15:36:52 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Result
of probe operation for p_ip_nfs_1 on cephmgr101.corp.mydomain.com: 7 (not
running)
Feb 22 15:36:52 cephmgr101.corp.mydomain.com
exportfs(p_exportfs_cart_sigtrack)[547]: ERROR:
/mnt/nfsroot/rbd0/cart_sigtrack does not exist or is not a directory
Feb 22 15:36:52 cephmgr101.corp.mydomain.com
exportfs(p_exportfs_cart_sigtrack)[553]: INFO: environment is invalid,
resource considered stopped
Feb 22 15:36:52 cephmgr101.corp.mydomain.com lrmd[32597]:   notice:
p_exportfs_cart_sigtrack_monitor_0:522:stderr [
ocf-exit-reason:/mnt/nfsroot/rbd0/cart_sigtrack does not exist or is not a
directory ]
Feb 22 15:36:52 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Result
of probe operation for p_exportfs_cart_sigtrack on
cephmgr101.corp.mydomain.com: 7 (not running)
Feb 22 15:36:52 cephmgr101.corp.mydomain.com crmd[32600]:   notice:
cephmgr101.corp.mydomain.com-p_exportfs_cart_sigtrack_monitor_0:27 [
ocf-exit-reason:/mnt/nfsroot/rbd0/cart_sigtrack does not exist or is not a
directory\n ]
Feb 22 15:36:55 cephmgr101.corp.mydomain.com kernel: libceph: mon1
172.20.3.22:6789 session established
Feb 22 15:36:55 cephmgr101.corp.mydomain.com kernel: libceph: client94252
fsid d36fd17c-174e-40d6-95b9-86bdd196b7d2
Feb 22 15:36:55 cephmgr101.corp.mydomain.com kernel: rbd: rbd0: capacity
8589934592 features 0x1
Feb 22 15:36:55 cephmgr101.corp.mydomain.com rbd.in(p_rbd_map_1)[587]:
INFO: /dev/rbd0
Feb 22 15:36:55 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Result
of start operation for p_rbd_map_1 on cephmgr101.corp.mydomain.com: 0 (ok)
Feb 22 15:36:55 cephmgr101.corp.mydomain.com Filesystem(p_fs_rbd_1)[655]:
INFO: Running start for /dev/rbd0 on /mnt/nfsroot/rbd0
Feb 22 15:36:55 cephmgr101.corp.mydomain.com kernel: XFS (rbd0): Mounting
V5 Filesystem
Feb 22 15:36:56 cephmgr101.corp.mydomain.com kernel: XFS (rbd0): Ending
clean mount
Feb 22 15:36:56 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Result
of start operation for p_fs_rbd_1 on cephmgr101.corp.mydomain.com: 0 (ok)
Feb 22 15:36:56 cephmgr101.corp.mydomain.com IPaddr2(p_ip_nfs_1)[791]:
INFO: Adding inet address 172.20.3.52/24 with broadcast address
172.20.3.255 to device ens160
Feb 22 15:36:56 cephmgr101.corp.mydomain.com IPaddr2(p_ip_nfs_1)[800]:
INFO: Bringing device ens160 up
Feb 22 15:36:56 cephmgr101.corp.mydomain.com IPaddr2(p_ip_nfs_1)[809]:
INFO: /usr/libexec/heartbeat/send_arp -i 200 -c 5 -p
/var/run/resource-agents/send_arp-172.20.3.52 -I ens160 -m auto 172.20.3.52
Feb 22 15:37:00 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Result
of start operation for p_ip_nfs_1 on cephmgr101.corp.mydomain.com: 0 (ok)
Feb 22 15:37:00 cephmgr101.corp.mydomain.com
exportfs(p_exportfs_cart_sigtrack)[872]: INFO: Directory
/mnt/nfsroot/rbd0/cart_sigtrack is not exported to 172.20.3.0/255.255.255.0
(stopped).
Feb 22 15:37:00 cephmgr101.corp.mydomain.com
exportfs(p_exportfs_cart_sigtrack)[887]: INFO: Exporting file system ...
Feb 22 15:37:00 cephmgr101.corp.mydomain.com
exportfs(p_exportfs_cart_sigtrack)[911]: INFO: exportfs: No file systems
exported! exporting 172.20.3.0/255.255.255.0:/mnt/nfsroot/rbd0/cart_sigtrack
Feb 22 15:37:00 cephmgr101.corp.mydomain.com
exportfs(p_exportfs_cart_sigtrack)[922]: INFO: File system exported
Feb 22 15:37:00 cephmgr101.corp.mydomain.com crmd[32600]:   notice: Result
of start operation for p_exportfs_cart_sigtrack on
cephmgr101.corp.mydomain.com: 0 (ok)


# ls -la /dev/rbd0
brw-rw---- 1 root disk 252, 0 Feb 22 15:36 /dev/rbd0


# mount | grep rbd0
/dev/rbd0 on /mnt/nfsroot/rbd0 type xfs
(rw,relatime,attr2,inode64,sunit=8192,swidth=8192,noquota)




Once I restart nfs-mountd, I see its not exporting the nfsroot:

# showmount -e
Export list for cephmgr101.corp.mydomain.com:
/mnt/nfsroot/rbd0/cart_sigtrack 172.20.3.0/255.255.255.0




# pcs resource show p_nfs_export_root_1
 Resource: p_nfs_export_root_1 (class=ocf provider=heartbeat type=exportfs)
  Attributes: clientspec=172.20.3.0/255.255.255.0 directory=/mnt/nfsroot
fsid=0 options=rw,crossmnt,async,no_root_squash
  Operations: monitor interval=30s
(p_nfs_export_root_1-monitor-interval-30s)
              start interval=0s timeout=40
(p_nfs_export_root_1-start-interval-0s)
              stop interval=0s timeout=120
(p_nfs_export_root_1-stop-interval-0s)




I've set the constraints the way I think is right, changing it doesn't seem
to help:
# pcs constraint order
Ordering Constraints:
  start p_rbd_map_1 then start p_nfs_export_root_1 (kind:Mandatory)
  start p_nfs_export_root_1 then start g_nfs_1 (kind:Mandatory)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20190222/20c91b73/attachment-0001.html>

More information about the Users mailing list