[ClusterLabs] Multiple nfsserver resource groups
christoforos at globalreach.com
Fri Mar 6 16:58:33 EST 2020
>I don't get it.
>Why the last nfs_shared_infodir will be mounted on /var/lib/nfs ?
>As far as I remember, you set that dir in any location you want as a Filesystem resource.
The dir you set in the resource options is indeed the dir used, but apparently the nfsserver daemon will mount that dir in /var/lib/nfs. I believe it's the last one because it's the last instance of the nfsdaemon that starts, so it "steals" the /var/lib/nfs mount from the other 2 that started before.
>From the ocf_heartbeat_nfsserver manpage ( https://manpages.debian.org/testing/resource-agents/ocf_heartbeat_nfsserver.7.en.html ) :
The nfsserver resource agent will save nfs related information in this specific directory. And this directory must be able to fail-over before nfsserver itself.
(optional, string, no default)
The mount point for the sunrpc file system. Default is /var/lib/nfs/rpc_pipefs. This script will mount (bind) nfs_shared_infodir on /var/lib/nfs/ (cannot be changed), and this script will mount the sunrpc file system on /var/lib/nfs/rpc_pipefs (default, can be changed by this parameter). If you want to move only rpc_pipefs/ (e.g. to keep rpc_pipefs/ local) from default, please set this value.
(optional, string, default "/var/lib/nfs/rpc_pipefs")"
>Note: You do realize that mpathi on 1 host can be turned into mpathZ on the other. Either use alias for each wwid, or do not use friendly names on those cluster nodes.
I know, wwid bindings are set to be identical on both nodes, so it's not an issue.
>Also, I would use LVM resource with 'exclusive=true' in order to avoid accidental activation of the LV on the other node.
I remember trying out LVM resource when I was doing initial testing but that was 2 years ago, so I can't remember why I went with a filesystem resource instead.
I will revisit that, thanks.
Any other insight on the nfs_shared_infodir situation?
From: Strahil Nikolov <hunter86_bg at yahoo.com>
Sent: Friday, March 6, 2020 11:20 PM
To: christoforos at globalreach.com; Cluster Labs - All topics related to open-source clustering welcomed <users at clusterlabs.org>
Subject: Re: [ClusterLabs] Multiple nfsserver resource groups
On March 6, 2020 7:56:00 PM GMT+02:00, Christoforos Christoforou <christoforos at globalreach.com> wrote:
>We have a PCS cluster running on 2 CentOS 7 nodes, exposing 2 NFSv3
>volumes which are then mounted to multiple servers (around 8).
>We want to have 2 more sets of additional shared NFS volumes, for a
>total of 6.
>I have successfully configured 3 resource groups, with each group
>having the following resources:
>* 1x ocf_heartbeat_IPaddr2 resource for the Virtual IP that exposes
>the NFS share assigned to its own NIC.
>* 3x ocf_heartbeat_Filesystem resources (1 is for the
>nfs_shared_infodir and the other 2 are the ones exposed via the NFS
>* 1x ocf_heartbeat_nfsserver resource that uses the aforementioned
>* 2x ocf_heartbeat_exportfs resources that expose the other 2
>filesystems as NFS shares.
>* 1x ocf_heartbeat_nfsnotify resource that has the Virtual IP set as
>its own source_host.
>All 9 filesystem volumes are mounted via iSCSI to the PCS nodes in
>So the structure is like so:
>Resource group 1:
>* /dev/mapper/mpatha - shared volume 1
>* /dev/mapper/mpathb - shared volume 2
>* /dev/mapper/mpathc - nfs_shared_infodir for resource group 1
>Resource group 2:
>* /dev/mapper/mpathd - shared volume 3
>* /dev/mapper/mpathe - shared volume 4
>* /dev/mapper/mpathf - nfs_shared_infodir for resource group 2
>Resource group 3:
>* /dev/mapper/mpathg - shared volume 5
>* /dev/mapper/mpathh - shared volume 6
>* /dev/mapper/mpathi - nfs_shared_infodir for resource group 3
>My concern is that when I run a df command on the active node, the last
>ocf_heartbeat_nfsserver volume (/dev/mapper/mpathi) mounted to
>I understand that I cannot change this, but I can change the location
>of the rpc_pipefs folder.
>I have had this setup running with 2 resource groups in our development
>environment, and have not noticed any issues, but since we're planning
>to move to production and add a 3rd resource group, I want to make sure
>that this setup will not cause any issues. I am by no means an expert
>on NFS, so some insight is appreciated.
>If this kind of setup is not supported or recommended, I have 2
>alternate plans in mind:
>1. Have all resources in the same resource group, in a setup that will
>look like this:
>a. 1x ocf_heartbeat_IPaddr2 resource for the Virtual IP that exposes
>the NFS share.
>b. 7x ocf_heartbeat_Filesystem resources (1 is for the
>nfs_shared_infodir and 6 exposed via the NFS server)
>c. 1x ocf_heartbeat_nfsserver resource that uses the aforementioned
>d. 6x ocf_heartbeat_exportfs resources that expose the other 6
>filesystems as NFS shares. Use the clientspec option to restrict to IPs
>and prevent unwanted mounts.
>e. 1x ocf_heartbeat_nfsnotify resource that has the Virtual IP set as
>its own source_host.
>2. Setup 2 more clusters to accommodate our needs
>I really want to avoid #2, due to the fact that it will be overkill for
>Senior Systems Administrator
>Global Reach Internet Productions
> <http://www.twitter.com/globalreach> Twitter |
><http://www.facebook.com/globalreach> Facebook |
>p (515) 996-0996 | <http://www.globalreach.com/> globalreach.com
I don't get it.
Why the last nfs_shared_infodir will be mounted on /var/lib/nfs ?
As far as I remember, you set that dir in any location you want as a Filesystem resource.
Note: You do realize that mpathi on 1 host can be turned into mpathZ on the other. Either use alias for each wwid, or do not use friendly names on those cluster nodes.
Also, I would use LVM resource with 'exclusive=true' in order to avoid accidental activation of the LV on the other node.
More information about the Users