[ClusterLabs] Fencing Approach

[Angelo Ruggiero]

Hello,

My setup....


  *
We are setting up a pacemaker cluster to run SAP runnig on RHEL on Vmware virtual machines.
  *
We will have two nodes for the application server of SAP and 2 nodes for the Hana database. SAP/RHEL provide good support on how to setup the cluster. 🙂
  *
SAP will need a number of floating Ips to be moved around as well mountin/unmounted NFS file system coming from a NetApp device. SAP will need processes switching on and off when something happens planned or unplanned.I am not clear if the netapp devic is active and the other site is DR but what i know is the ip addresses just get moved during a DR incident. Just to be complete the HANA data sync is done by HANA itself most probably async with an RPO of 15mins or so.
  *
 We will have a quorum node also with hopefully a seperate network not sure if it will be on a seperate vmware infra though.
  *
I am hoping to be allowed to use the vmware watchdog although it might take some persuading as declared as "non standard" for us by our infra people. I have it already in DEV to play with now.

I managed to set the above working just using a floating ip and a nfs mount as my resources and I can see the following. The self fencing approach works fine i.e the servers reboot when they loose network connectivity and/or become in quorate as long as they are offering resources.

So my questions are in relation to further fencing .... I did a lot of reading and saw various reference...


  1.
Use of sbd shared storage

The question is what does using sbd with a shated storage really give me. I need to justify why i need this shared storage again to the infra guys but to be honest also to myself.   I have been given this infra and will play with it next few days.


  1.
Use of fence vmware

In addition there is the ability of course to fence using the fence_vmware agents and I again I need to justify why i need this. In this particular cases it will be a very hard sell because the dev/test and prod environments run on the same vmware infra so to use fence_vmware would effectively mean dev is connected to prod i.e the user id for a dev or test box is being provided by a production environment. I do not have this ability at all so cannot play with it.



My current thought train...i.e the typical things i think about...

Perhaps someone can help me be clear on the benefits of 1 and 2 over and above the setup i think it doable.


  1.
 gives me the ability to use poison pill

But what scenarios does poison pill really help why would the other parts of the cluster want to fence the node if the node itself has not killed it self as it lost quorum either because quorum devcice gone or network connectivity failed and resources needs to be switched off.

              What i get is that it is very explict i.e the others nodes tell the other server to die. So it must be a case initiated by the other nodes.
              I am struggling to think of a scenarios where the other nodes would want to fence it.

Possible Scenarios, did i miss any?

  *
Loss of network connection to the node. But that is covered by the node self fencing
  *
If some monitoring said the node was not healthly or responding... Maybe this is the case it is good for but then it must be a partial failure where the node is still part fof the cluster and can respond. I.e not OS freeze or only it looses connection as then the watchdog or the self fencing will kick in.
  *
HW failures, cpu, memory, disk For virtual hardware does that actually ever fail? Sorry if stupid question. I could ask our infra guys but....,
So is virtual hardware so reliable that hw failures can be ignored.
  *
Loss of shared storage SAP uses a lot of shared storage via NFS. Not sure what happens when that fails need to research it a bit but each node will sort that out itself I am presuming.
  *
Human error: but no cluster will fix that and the human who makes a change will realise it and revert. 🙂

       2. Fence vmware

      I see this as a better poision pill as it works at the hardware level. But if I do not need poision pill then i do not need this.

In general OS freezes or even panics if take took long are covered by the watchdog.

regards
Angelo





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20241009/62632711/attachment-0001.htm>