[ClusterLabs] Two-node cluster fencing

[Ken Gaillot]

On Sun, 2018-05-13 at 22:58 +0800, Confidential Company wrote:
> Hi Casey,
> 
> 1. I tried shutting down my VM while testing, the "ClusterIP"
> resource switched automatically onto the standy node(Node2)
> 2. I do "systemctl enable corosnyc/pacemaker" - so that after reboot,
> corosync and pacemaker will automatically start.
> 3. As I turn-on Node1, I experienced downtime (maybe syncing of nodes
> will result of downtime), but my cluster still works as expected -->
> The active node is still Node2.

There shouldn't be any downtime directly as a result of a node joining.
However if you didn't set any resource stickiness, it's possible the
cluster moved the IP back to the newly joined node.

> 4. If I choose ESXI as my fence device, if the physical server goes
> down, would it still be reasonable because its on one host?

I'm not too familiar with ESXI so hopefully someone else can comment on
the specifics there, but in general, with VM fencing, you do still have
an issue if the hypervisor that the fence device talks to doesn't
respond for whatever reason. If the cluster can't get a confirmation
that the VM was fenced, it won't recover any resources from it.

One way around this is a fencing topology i.e. a fallback device. You
would use VM fencing as the first topology level, so that if it
succeeds, that's sufficient, but if it fails, a second topology level
(e.g. a power strip connected to the physical machine) would be tried.

> 
> Thanks Casey, I want to understand more about fencing. 
> 
> 
> Regards,
> 
> imnotarobot
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> >>Without fencing, if the primary is powered off abruptly (e.g. if
> one of your ESX servers crashes), the standby will not become
> primary, and you will need to promote it manually.  We had exactly
> this scenario happen last week with a 2-node cluster.  Without
> fencing, you don't have high availability.  If you don't need high
> availability, you probably don't need pacemaker.
> 
> There are instructions for setting up fencing with vmware here:  http
> s://www.hastexo.com/resources/hints-and-kinks/fencing-vmware-
> virtualized-pacemaker-nodes/
> 
> One note - rather than the SDK, I believe you actually need the CLI
> package, which can be found here:  https://my.vmware.com/web/vmware/d
> etails?downloadGroup=VCLI600&productId=491
> 
> Good luck - I haven't managed to get it to build yet - vmware gives
> you a black box installer script that compiles a bunch of dependent
> perl modules, and it ends up getting hung with 100% CPU usage for
> days - digging into this further with lsof and friends, it seems to
> be prompting for where your apache source code is to compile
> mod_perl.  Why does it need mod_perl for the CLI??  Anyways, I
> haven't managed to get past that roadblock yet.  I'm using Ubuntu 16
> so it may happen to just work better on your RHEL instances.  If you
> have a different ESX version than 6.0, you may have better luck as
> well.
> 
> Best wishes,
> -- 
> Casey
> 
> > On May 11, 2018, at 10:31 PM, Confidential Company <sgurovosa at gmail
> .com> wrote:
> > 
> > Hi,
> > 
> > This is my setup:
> > 
> > 1. I have Two vMware-ESXI hosts with one virtual machine (RHEL 7.4)
> on each.
> > 2. On my physical machine, I have four vmnic --> vmnic 0,1 for
> uplink going to switchA and switchB --> vmnic 2,3 for heartbeat
> corosync traffic (direct connect to other ESXI host)
> > 3. I plan on clustering my two virtual machines via corosync and
> create a virtual-IP via pacemaker. 
> > 4. I plan on using the uplink interface for data and totem
> interface for corosync packets(heartbeat messages).
> > 5. These two virtual machines doesnt need for a shared storage, or
> a shared LUN because the application is, by nature, a standalone
> application that doesnt need to have a centralized location as it
> does not store any data that needs to be synchronized between two
> servers.
> > 6. I have a PC that only needs to contact the Virtual IP of the
> rhel virtual servers.
> > 7. Seamless failover from primary to secondary is not required. 
> > 8. Active/Passive setup
> > 
> > 
> > Given the setup above, 
> > 1. Is there any drawbacks?
> > 2. Do I need fencing? Can you explain me by giving a scenario on
> the above setup? What instances will occur if I didnt put a fence
> device?
> > 3. If I need a fence device? what fence device you recommend? SAN,
> vmWare, or PDU?
> > 
> > 
> > Thanks,
> > 
> > imnotarobot
-- 
Ken Gaillot <kgaillot at redhat.com>