[ClusterLabs] Still Beginner STONITH Problem

[Klaus Wenninger]

On 7/14/20 10:06 AM, stefan.schmitz at farmpartner-tec.com wrote:
> Hello,
>
>
> Am 09.07.2020 um 19:10 Strahil Nikolov wrote:
> >Have  you  run 'fence_virtd  -c' ?
> Yes I had run that on both Hosts. The current config looks like that
> and is identical on both.
>
> cat fence_virt.conf
> fence_virtd {
>         listener = "multicast";
>         backend = "libvirt";
>         module_path = "/usr/lib64/fence-virt";
> }
>
> listeners {
>         multicast {
>                 key_file = "/etc/cluster/fence_xvm.key";
>                 address = "225.0.0.12";
>                 interface = "bond0";
>                 family = "ipv4";
>                 port = "1229";
>         }
>
> }
>
> backends {
>         libvirt {
>                 uri = "qemu:///system";
>         }
>
> }
>
>
> The situation is still that no matter on what host I issue the
> "fence_xvm -a 225.0.0.12 -o list" command, both guest systems receive
> the traffic. The local guest, but also the guest on the other host. I
> reckon that means the traffic is not filtered by any network device,
> like switches or firewalls. Since the guest on the other host receives
> the packages, the traffic must reach te physical server and
> networkdevice and is then routed to the VM on that host.
> But still, the traffic is not shown on the host itself.
>
> Further the local firewalls on both hosts are set to let each and
> every traffic pass. Accept to any and everything. Well at least as far
> as I can see.
>
>
> Am 09.07.2020 um 22:34 Klaus Wenninger wrote:
> > makes me believe that
> > the whole setup doesn't lookas I would have
> > expected (bridges on each host where theguest
> > has a connection to and where ethernet interfaces
> > that connect the 2 hosts are part of as well
>
> On each physical server the networkcards are bonded to achieve failure
> safety (bond0). The guest are connected over a bridge(br0) but
> apparently our virtualization softrware creates an own device named
> after the guest (kvm101.0).
> There is no direct connection between the servers, but as I said
> earlier, the multicast traffic does reach the VMs so I assume there is
> no problem with that.
Guess it is not easy to have your servers connected physically for a try.
But maybe you can at least try on one host to have virt_fenced & VM
on the same bridge - just to see if that basic pattern is working.
>
>
> Am 09.07.2020 um 20:18 Vladislav Bogdanov wrote:
> > First, you need to ensure that your switch (or all switches in the
> > path) have igmp snooping enabled on host ports (and probably
> > interconnects along the path between your hosts).
> >
> > Second, you need an igmp querier to be enabled somewhere near (better
> > to have it enabled on a switch itself). Please verify that you see its
> > queries on hosts.
> >
> > Next, you probably need to make your hosts to use IGMPv2 (not 3) as
> > many switches still can not understand v3. This is doable by sysctl,
> > find on internet, there are many articles.
>
>
> I have send an query to our Data center Techs who are analyzing this
> and were already on it analyzing if multicast Traffic is somewhere
> blocked or hindered. So far the answer is, "multicast ist explictly
> allowed in the local network and no packets are filtered or dropped".
> I am still waiting for a final report though.
>
> In the meantime I have switched IGMPv3 to IGMPv2 on every involved
> server, hosts and guests via the mentioned sysctl. The switching
> itself was successful, according to "cat /proc/net/igmp" but sadly did
> not better the behavior. It actually led to that no VM received the
> multicast traffic anymore too.
Well maybe still sbdy in the middle playing IGMPv3 or the request for
a certain source is needed to shoot open some firewall or switch-tables.
>
> kind regards
> Stefan Schmitz
>
>
> Am 09.07.2020 um 22:34 schrieb Klaus Wenninger:
>> On 7/9/20 5:17 PM, stefan.schmitz at farmpartner-tec.com wrote:
>>> Hello,
>>>
>>>> Well, theory still holds I would say.
>>>>
>>>> I guess that the multicast-traffic from the other host
>>>> or the guestsdoesn't get to the daemon on the host.
>>>> Can't you just simply check if there are any firewall
>>>> rules configuredon the host kernel?
>>>
>>> I hope I did understand you corretcly and you are referring to
>>> iptables?
>> I didn't say iptables because it might have been
>> nftables - but yesthat is what I was referring to.
>> Guess to understand the config the output is
>> lacking verbositybut it makes me believe that
>> the whole setup doesn't lookas I would have
>> expected (bridges on each host where theguest
>> has a connection to and where ethernet interfaces
>> that connect the 2 hosts are part of as well -
>> everythingconnected via layer 2 basically).
>>> Here is the output of the current rules. Besides the IP of the guest
>>> the output is identical on both hosts:
>>>
>>> # iptables -S
>>> -P INPUT ACCEPT
>>> -P FORWARD ACCEPT
>>> -P OUTPUT ACCEPT
>>>
>>> # iptables -L
>>> Chain INPUT (policy ACCEPT)
>>> target     prot opt source               destination
>>>
>>> Chain FORWARD (policy ACCEPT)
>>> target     prot opt source               destination
>>> SOLUSVM_TRAFFIC_IN  all  --  anywhere             anywhere
>>> SOLUSVM_TRAFFIC_OUT  all  --  anywhere             anywhere
>>>
>>> Chain OUTPUT (policy ACCEPT)
>>> target     prot opt source               destination
>>>
>>> Chain SOLUSVM_TRAFFIC_IN (1 references)
>>> target     prot opt source               destination
>>>             all  --  anywhere             192.168.1.14
>>>
>>> Chain SOLUSVM_TRAFFIC_OUT (1 references)
>>> target     prot opt source               destination
>>>             all  --  192.168.1.14         anywhere
>>>
>>> kind regards
>>> Stefan Schmitz
>>>
>>>
>>
>