<div dir="auto">Hello Ken,<div dir="auto"><br></div><div dir="auto">Thank you. </div><div dir="auto"><br></div><div dir="auto">But if I have a two node cluster and a working fencing mechanism wouldn't it be enough to disable the corosync and pacemaker service on both nodes so when it fence they won't come up?</div><div dir="auto"><br></div><div dir="auto">Thank you</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">pon., 18.03.2019, 16:19 użytkownik Ken Gaillot <<a href="mailto:kgaillot@redhat.com">kgaillot@redhat.com</a>> napisał:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Sat, 2019-03-16 at 11:10 +0100, Adam Budziński wrote:<br>
> Hello Andrei,<br>
> <br>
> Ok I see your point. So per my understanding if the resource is<br>
> started successfully in that case fence vmware it will be monitored<br>
> indefinitely but as you sad it will monitor the current active node.<br>
> So how does the fence agent gets aware of problems with the slave? I <br>
<br>
The fence agent doesn't monitor the active node, or any node -- it<br>
monitors the fence device.<br>
<br>
The cluster layer (i.e. corosync) monitors all nodes, and reports any<br>
issues to pacemaker, which will initiate fencing if necessary.<br>
<br>
Pacemaker also monitors each resource and fence device, via any<br>
recurring monitors that have been configured.<br>
<br>
> mean if in a two node cluster the cluster splits in to two partitions<br>
> each of them will fence the other or does that happen because both<br>
> will assume they are the only survivors and thus need to fence the<br>
> other node which is in a unknow state so to say?<br>
<br>
If both nodes are functional but can't see each other, they will each<br>
want to initiate fencing. If one of them is quicker than the other to<br>
determine this, the other one will get shot before it has a chance to<br>
do anything itself.<br>
<br>
However there is the possibility that both nodes will shoot at about<br>
the same time, resulting in both nodes getting shot (a "stonith death<br>
match"). This is only a problem in 2-node clusters. There are a few<br>
ways around this:<br>
<br>
1. Configure two separate fence devices, each targeting one of the<br>
nodes, and put a delay on one of them (or a random delay on both). This<br>
makes it highly unlikely that they will shoot at the same time.<br>
<br>
2. Configure a fencing topology with a fence heuristics device plus<br>
your real device. A fence heuristics device runs some test, and refuses<br>
to shoot the other node if the test fails. For example,<br>
fence_heuristics_ping tries to ping an IP address you give it; the idea<br>
is that if a node can't ping that IP, you don't want it to survive.<br>
This ensures that only a node that passes the test can shoot (which<br>
means there still might be some cases where the nodes can both shoot<br>
each other, and cases where the cluster will freeze because neither<br>
node can see the IP).<br>
<br>
3. Configure corosync with qdevice to provide true quorum via a third<br>
host (which doesn't participate in the cluster otherwise).<br>
<br>
4. Use sbd with a hardware watchdog and a shared storage device as the<br>
fencing device. This is not a reliable option with VMWare, but I'm<br>
listing it for the general case.<br>
<br>
<br>
> <br>
> Thank you and Best Regards,<br>
> Adam <br>
> <br>
> sob., 16.03.2019, 07:17 użytkownik Andrei Borzenkov <<br>
> <a href="mailto:arvidjaar@gmail.com" target="_blank" rel="noreferrer">arvidjaar@gmail.com</a>> napisał:<br>
> > 16.03.2019 9:01, Adam Budziński пишет:<br>
> > > Thank you Andrei. The problem is that I can see with 'pcs status'<br>
> > that<br>
> > > resources are runnin on srv2cr1 but its at the same time its<br>
> > telling that<br>
> > > the fence_vmware_soap is running on srv1cr1. That's somewhat<br>
> > confusing.<br>
> > > Could you possibly explain this?<br>
> > > <br>
> > <br>
> > Two points.<br>
> > <br>
> > It is actually logical to have stonith agent running on different<br>
> > node<br>
> > than node with active resources - because it is the *other* node<br>
> > that<br>
> > will initiate fencing when node with active resources fails.<br>
> > <br>
> > But even considering the above, active (running) state of fence (or<br>
> > stonith) agent just determines on which node recurring monitor<br>
> > operation<br>
> > will be started. The actual result of this monitor operation has no<br>
> > impact on subsequent stonith attempt and serves just as warning to<br>
> > administrator. When stonith request comes, agent may be used by any<br>
> > node<br>
> > where stonith agent is not prohibited to run by (co-)location<br>
> > rules. My<br>
> > understanding is that this node is selected by DC in partition.<br>
> > <br>
> > > Thank you!<br>
> > > <br>
> > > sob., 16.03.2019, 05:37 użytkownik Andrei Borzenkov <<br>
> > <a href="mailto:arvidjaar@gmail.com" target="_blank" rel="noreferrer">arvidjaar@gmail.com</a>><br>
> > > napisał:<br>
> > > <br>
> > >> 16.03.2019 1:16, Adam Budziński пишет:<br>
> > >>> Hi Tomas,<br>
> > >>><br>
> > >>> Ok but how then pacemaker or the fence agent knows which route<br>
> > to take to<br>
> > >>> reach the vCenter?<br>
> > >><br>
> > >> They do not know or care at all. It is up to your underlying<br>
> > operating<br>
> > >> system and its routing tables.<br>
> > >><br>
> > >>> Btw. Do I have to add the stonith resource on each of the nodes<br>
> > or is it<br>
> > >>> just enough to add it on one as for other resources?<br>
> > >><br>
> > >> If your fencing agent can (should) be able to run on any node,<br>
> > it should<br>
> > >> be enough to define it just once as long as it can properly<br>
> > determine<br>
> > >> "port" to use on fencing "device" for a given node. There are<br>
> > cases when<br>
> > >> you may want to restrict fencing agent to only subset on nodes<br>
> > or when<br>
> > >> you are forced to set unique parameter for each node (consider<br>
> > IPMI IP<br>
> > >> address), in this case you would need separate instance of agent<br>
> > in each<br>
> > >> case.<br>
> > >><br>
> > >>> Thank you!<br>
> > >>><br>
> > >>> pt., 15.03.2019, 15:48 użytkownik Tomas Jelinek <<br>
> > <a href="mailto:tojeline@redhat.com" target="_blank" rel="noreferrer">tojeline@redhat.com</a>><br>
> > >>> napisał:<br>
> > >>><br>
> > >>>> Dne 15. 03. 19 v 15:09 Adam Budziński napsal(a):<br>
> > >>>>> Hello Tomas,<br>
> > >>>>><br>
> > >>>>> Thank you! So far I need to say how great this community is,<br>
> > would<br>
> > >>>>> never expect so much positive vibes! A big thank you your<br>
> > doing a great<br>
> > >>>>> job!<br>
> > >>>>><br>
> > >>>>> Now let's talk business :)<br>
> > >>>>><br>
> > >>>>> So if pcsd is using ring0 and it fails will ring1 not be used<br>
> > at all?<br>
> > >>>><br>
> > >>>> Pcs and pcsd never use ring1, but they are just tools for<br>
> > managing<br>
> > >>>> clusters. You can have a perfectly functioning cluster without<br>
> > pcs and<br>
> > >>>> pcsd running or even installed, it would be just more<br>
> > complicated to set<br>
> > >>>> it up and manage it.<br>
> > >>>><br>
> > >>>> Even if ring0 fails, you will be able to use pcs (in somehow<br>
> > limited<br>
> > >>>> manner) as most of its commands don't go through network<br>
> > anyway.<br>
> > >>>><br>
> > >>>> Corosync, which is the actual cluster messaging layer, will of<br>
> > course<br>
> > >>>> use ring1 in case of ring0 failure.<br>
> > >>>><br>
> > >>>>><br>
> > >>>>> So in regards to VMware that would mean that the interface<br>
> > should be<br>
> > >>>>> configured with a network that can access the vCenter to<br>
> > fence right?<br>
> > >>>>> But wouldn't it then use only ring0 so if that fails it<br>
> > wouldn't switch<br>
> > >>>>> to ring1?<br>
> > >>>><br>
> > >>>> If you are talking about pcmk_host_map, that does not really<br>
> > have<br>
> > >>>> anything to do with network interfaces of cluster nodes. It<br>
> > maps node<br>
> > >>>> names (parts before :) to "ports" of a fence device (parts<br>
> > after :).<br>
> > >>>> Pcs-0.9.x does not support defining custom node names,<br>
> > therefore node<br>
> > >>>> names are the same as ring0 addresses.<br>
> > >>>><br>
> > >>>> I am not an expert on fence agents / devices, but I'm sure<br>
> > someone else<br>
> > >>>> on this list will be able to help you with configuring fencing<br>
> > for your<br>
> > >>>> cluster.<br>
> > >>>><br>
> > >>>><br>
> > >>>> Tomas<br>
> > >>>><br>
> > >>>>><br>
> > >>>>> Thank you!<br>
> > >>>>><br>
> > >>>>> pt., 15.03.2019, 13:14 użytkownik Tomas Jelinek <<br>
> > <a href="mailto:tojeline@redhat.com" target="_blank" rel="noreferrer">tojeline@redhat.com</a><br>
> > >>>>> <mailto:<a href="mailto:tojeline@redhat.com" target="_blank" rel="noreferrer">tojeline@redhat.com</a>>> napisał:<br>
> > >>>>><br>
> > >>>>> Dne 15. 03. 19 v 12:32 Adam Budziński napsal(a):<br>
> > >>>>> > Hello Folks,____<br>
> > >>>>> ><br>
> > >>>>> > __ __<br>
> > >>>>> ><br>
> > >>>>> > Tow node active/passive VMware VM cluster.____<br>
> > >>>>> ><br>
> > >>>>> > __ __<br>
> > >>>>> ><br>
> > >>>>> > /etc/hosts____<br>
> > >>>>> ><br>
> > >>>>> > __ __<br>
> > >>>>> ><br>
> > >>>>> > 10.116.63.83 srv1____<br>
> > >>>>> ><br>
> > >>>>> > 10.116.63.84 srv2____<br>
> > >>>>> ><br>
> > >>>>> > 172.16.21.12 srv2cr1____<br>
> > >>>>> ><br>
> > >>>>> > 172.16.22.12 srv2cr2____<br>
> > >>>>> ><br>
> > >>>>> > 172.16.21.11 srv1cr1____<br>
> > >>>>> ><br>
> > >>>>> > 172.16.22.11 srv1cr2____<br>
> > >>>>> ><br>
> > >>>>> > __ __<br>
> > >>>>> ><br>
> > >>>>> > __ __<br>
> > >>>>> ><br>
> > >>>>> > I have 3 NIC’s on each VM:____<br>
> > >>>>> ><br>
> > >>>>> > __ __<br>
> > >>>>> ><br>
> > >>>>> > 10.116.63.83 srv1 and 10.116.63.84 srv2 are<br>
> > networks used<br>
> > >>>> to<br>
> > >>>>> > access the VM’s via SSH or any resource directly if<br>
> > not via a<br>
> > >>>>> VIP.____<br>
> > >>>>> ><br>
> > >>>>> > __ __<br>
> > >>>>> ><br>
> > >>>>> > Everything with cr in its name is used for corosync<br>
> > >>>>> communication, so<br>
> > >>>>> > basically I have two rings (this are two no routable<br>
> > networks<br>
> > >>>>> just for<br>
> > >>>>> > that).____<br>
> > >>>>> ><br>
> > >>>>> > __ __<br>
> > >>>>> ><br>
> > >>>>> > My questions are:____<br>
> > >>>>> ><br>
> > >>>>> > __ __<br>
> > >>>>> ><br>
> > >>>>> > __1.__With ‘pcs cluster auth’ which interface /<br>
> > interfaces<br>
> > >> should<br>
> > >>>>> I use<br>
> > >>>>> > ?____<br>
> > >>>>><br>
> > >>>>> Hi Adam,<br>
> > >>>>><br>
> > >>>>> I can see you are using pcs-0.9.x. In that case you<br>
> > should do:<br>
> > >>>>> pcs cluster auth srv1cr1 srv2cr1<br>
> > >>>>><br>
> > >>>>> In other words, use the first address of each node.<br>
> > >>>>> Authenticating all the other addresses should not cause<br>
> > any issues.<br>
> > >>>> It<br>
> > >>>>> is pointless, though, as pcs only communicates via ring0<br>
> > addresses.<br>
> > >>>>><br>
> > >>>>> ><br>
> > >>>>> > __2.__With ‘pcs cluster setup –name’ I would use the<br>
> > corosync<br>
> > >>>>> interfaces<br>
> > >>>>> > e.g. ‘pcs cluster setup –name MyCluster<br>
> > srv1cr1,srv1cr2<br>
> > >>>>> srv2cr1,srv2cr2’<br>
> > >>>>> > right ?____<br>
> > >>>>><br>
> > >>>>> Yes, that is correct.<br>
> > >>>>><br>
> > >>>>> ><br>
> > >>>>> > __3.__With fence_vmware_soap<br>
> > >> inpcmk_host_map="X:VM_C;X:VM:OTRS_D"<br>
> > >>>>> which<br>
> > >>>>> > interface should replace X ?____<br>
> > >>>>><br>
> > >>>>> X should be replaced by node names as seen by pacemaker.<br>
> > Once you<br>
> > >>>>> set up<br>
> > >>>>> and start your cluster, run 'pcs status' to get (amongs<br>
> > other info)<br>
> > >>>> the<br>
> > >>>>> node names. In your configuration, they should be srv1cr1<br>
> > and<br>
> > >>>> srv2cr1.<br>
> > >>>>><br>
> > >>>>><br>
> > >>>>> Regards,<br>
> > >>>>> Tomas<br>
> > >>>>><br>
> > >>>>> > __ __<br>
> > >>>>> ><br>
> > >>>>> > Thank you!<br>
> > >>>>> ><br>
> > >>>>> ><br>
> > >>>>> > _______________________________________________<br>
> > >>>>> > Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a><br>
> > >>>>> <mailto:<a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a>><br>
> > >>>>> > <a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
> > >>>>> ><br>
> > >>>>> > Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
> > >>>>> > Getting started:<br>
> > >>>>> <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > >>>>> > Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
> > >>>>> ><br>
> > >>>>> _______________________________________________<br>
> > >>>>> Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a> <mailto:<br>
> > >>>> <a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a>><br>
> > >>>>> <a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
> > >>>>><br>
> > >>>>> Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
> > >>>>> Getting started:<br>
> > >>>> <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > >>>>> Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
> > >>>>><br>
> > >>>>><br>
> > >>>>> _______________________________________________<br>
> > >>>>> Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a><br>
> > >>>>> <a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
> > >>>>><br>
> > >>>>> Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
> > >>>>> Getting started:<br>
> > >> <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > >>>>> Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
> > >>>>><br>
> > >>>> _______________________________________________<br>
> > >>>> Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a><br>
> > >>>> <a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
> > >>>><br>
> > >>>> Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
> > >>>> Getting started:<br>
> > >> <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > >>>> Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
> > >>>><br>
> > >>><br>
> > >>><br>
> > >>> _______________________________________________<br>
> > >>> Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a><br>
> > >>> <a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
> > >>><br>
> > >>> Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
> > >>> Getting started: <br>
> > <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > >>> Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
> > >>><br>
> > >><br>
> > >> _______________________________________________<br>
> > >> Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a><br>
> > >> <a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
> > >><br>
> > >> Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
> > >> Getting started: <br>
> > <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > >> Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
> > >><br>
> > > <br>
> > > <br>
> > > _______________________________________________<br>
> > > Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a><br>
> > > <a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
> > > <br>
> > > Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
> > > Getting started: <br>
> > <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > > Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
> > > <br>
> > <br>
> > _______________________________________________<br>
> > Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a><br>
> > <a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
> > <br>
> > Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
> > Getting started: <br>
> > <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> > Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
> <br>
> _______________________________________________<br>
> Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a><br>
> <a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
> <br>
> Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
> Getting started: <br>
> <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
-- <br>
Ken Gaillot <<a href="mailto:kgaillot@redhat.com" target="_blank" rel="noreferrer">kgaillot@redhat.com</a>><br>
<br>
_______________________________________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank" rel="noreferrer">Users@clusterlabs.org</a><br>
<a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer noreferrer" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
</blockquote></div>