[ClusterLabs] Still Beginner STONITH Problem
Klaus Wenninger
kwenning at redhat.com
Wed Jul 15 06:19:59 EDT 2020
On 7/15/20 11:42 AM, stefan.schmitz at farmpartner-tec.com wrote:
> Hello,
>
>
> Am 15.07.2020 um 06:32 Strahil Nikolov wrote:
>> How did you configure the network on your ubuntu 20.04 Hosts ? I
>> tried to setup bridged connection for the test setup , but obviously
>> I'm missing something.
>>
>> Best Regards,
>> Strahil Nikolov
>>
>
> on the hosts (CentOS) the bridge config looks like that.The bridging
> and configuration is handled by the virtualization software:
>
> # cat ifcfg-br0
> DEVICE=br0
> TYPE=Bridge
> BOOTPROTO=static
> ONBOOT=yes
> IPADDR=192.168.1.21
> NETMASK=255.255.0.0
> GATEWAY=192.168.1.1
> NM_CONTROLLED=no
> IPV6_AUTOCONF=yes
> IPV6_DEFROUTE=yes
> IPV6_PEERDNS=yes
> IPV6_PEERROUTES=yes
> IPV6_FAILURE_FATAL=no
>
>
>
> Am 15.07.2020 um 09:50 Klaus Wenninger wrote:
> > Guess it is not easy to have your servers connected physically for a
> try.
> > But maybe you can at least try on one host to have virt_fenced & VM
> > on the same bridge - just to see if that basic pattern is working.
>
> I am not sure if I understand you correctly. What do you by having
> them on the same bridge? The bridge device is configured on the host
> by the virtualization software.
I meant to check out which bridge the interface of the VM is enslaved
to and to use that bridge as interface in /etc/fence_virt.conf.
Get me right - just for now - just to see if it is working for this one
host and the corresponding guest.
>
>
> >Well maybe still sbdy in the middle playing IGMPv3 or the request for
> >a certain source is needed to shoot open some firewall or switch-tables.
>
> I am still waiting for the final report from our Data Center techs. I
> hope that will clear up somethings.
>
>
> Additionally I have just noticed that apparently since switching from
> IGMPv3 to IGMPv2 and back the command "fence_xvm -a 225.0.0.12 -o
> list" is no completely broken.
> Before that switch this command at least returned the local VM. Now it
> returns:
> Timed out waiting for response
> Operation failed
>
> I am a bit confused by that, because all we did was running commands
> like "sysctl -w net.ipv4.conf.all.force_igmp_version =" with the
> different Version umbers and #cat /proc/net/igmp shows that V3 is used
> again on every device just like before...?!
>
> kind regards
> Stefan Schmitz
>
>
>> На 14 юли 2020 г. 11:06:42 GMT+03:00,
>> "stefan.schmitz at farmpartner-tec.com"
>> <stefan.schmitz at farmpartner-tec.com> написа:
>>> Hello,
>>>
>>>
>>> Am 09.07.2020 um 19:10 Strahil Nikolov wrote:
>>>> Have you run 'fence_virtd -c' ?
>>> Yes I had run that on both Hosts. The current config looks like that
>>> and
>>> is identical on both.
>>>
>>> cat fence_virt.conf
>>> fence_virtd {
>>> listener = "multicast";
>>> backend = "libvirt";
>>> module_path = "/usr/lib64/fence-virt";
>>> }
>>>
>>> listeners {
>>> multicast {
>>> key_file = "/etc/cluster/fence_xvm.key";
>>> address = "225.0.0.12";
>>> interface = "bond0";
>>> family = "ipv4";
>>> port = "1229";
>>> }
>>>
>>> }
>>>
>>> backends {
>>> libvirt {
>>> uri = "qemu:///system";
>>> }
>>>
>>> }
>>>
>>>
>>> The situation is still that no matter on what host I issue the
>>> "fence_xvm -a 225.0.0.12 -o list" command, both guest systems receive
>>> the traffic. The local guest, but also the guest on the other host. I
>>> reckon that means the traffic is not filtered by any network device,
>>> like switches or firewalls. Since the guest on the other host receives
>>> the packages, the traffic must reach te physical server and
>>> networkdevice and is then routed to the VM on that host.
>>> But still, the traffic is not shown on the host itself.
>>>
>>> Further the local firewalls on both hosts are set to let each and every
>>>
>>> traffic pass. Accept to any and everything. Well at least as far as I
>>> can see.
>>>
>>>
>>> Am 09.07.2020 um 22:34 Klaus Wenninger wrote:
>>>> makes me believe that
>>>> the whole setup doesn't lookas I would have
>>>> expected (bridges on each host where theguest
>>>> has a connection to and where ethernet interfaces
>>>> that connect the 2 hosts are part of as well
>>>
>>> On each physical server the networkcards are bonded to achieve failure
>>> safety (bond0). The guest are connected over a bridge(br0) but
>>> apparently our virtualization softrware creates an own device named
>>> after the guest (kvm101.0).
>>> There is no direct connection between the servers, but as I said
>>> earlier, the multicast traffic does reach the VMs so I assume there is
>>> no problem with that.
>>>
>>>
>>> Am 09.07.2020 um 20:18 Vladislav Bogdanov wrote:
>>>> First, you need to ensure that your switch (or all switches in the
>>>> path) have igmp snooping enabled on host ports (and probably
>>>> interconnects along the path between your hosts).
>>>>
>>>> Second, you need an igmp querier to be enabled somewhere near (better
>>>> to have it enabled on a switch itself). Please verify that you see
>>> its
>>>> queries on hosts.
>>>>
>>>> Next, you probably need to make your hosts to use IGMPv2 (not 3) as
>>>> many switches still can not understand v3. This is doable by sysctl,
>>>> find on internet, there are many articles.
>>>
>>>
>>> I have send an query to our Data center Techs who are analyzing this
>>> and
>>> were already on it analyzing if multicast Traffic is somewhere blocked
>>> or hindered. So far the answer is, "multicast ist explictly allowed in
>>> the local network and no packets are filtered or dropped". I am still
>>> waiting for a final report though.
>>>
>>> In the meantime I have switched IGMPv3 to IGMPv2 on every involved
>>> server, hosts and guests via the mentioned sysctl. The switching itself
>>>
>>> was successful, according to "cat /proc/net/igmp" but sadly did not
>>> better the behavior. It actually led to that no VM received the
>>> multicast traffic anymore too.
>>>
>>> kind regards
>>> Stefan Schmitz
>>>
>>>
>>> Am 09.07.2020 um 22:34 schrieb Klaus Wenninger:
>>>> On 7/9/20 5:17 PM, stefan.schmitz at farmpartner-tec.com wrote:
>>>>> Hello,
>>>>>
>>>>>> Well, theory still holds I would say.
>>>>>>
>>>>>> I guess that the multicast-traffic from the other host
>>>>>> or the guestsdoesn't get to the daemon on the host.
>>>>>> Can't you just simply check if there are any firewall
>>>>>> rules configuredon the host kernel?
>>>>>
>>>>> I hope I did understand you corretcly and you are referring to
>>> iptables?
>>>> I didn't say iptables because it might have been
>>>> nftables - but yesthat is what I was referring to.
>>>> Guess to understand the config the output is
>>>> lacking verbositybut it makes me believe that
>>>> the whole setup doesn't lookas I would have
>>>> expected (bridges on each host where theguest
>>>> has a connection to and where ethernet interfaces
>>>> that connect the 2 hosts are part of as well -
>>>> everythingconnected via layer 2 basically).
>>>>> Here is the output of the current rules. Besides the IP of the guest
>>>>> the output is identical on both hosts:
>>>>>
>>>>> # iptables -S
>>>>> -P INPUT ACCEPT
>>>>> -P FORWARD ACCEPT
>>>>> -P OUTPUT ACCEPT
>>>>>
>>>>> # iptables -L
>>>>> Chain INPUT (policy ACCEPT)
>>>>> target prot opt source destination
>>>>>
>>>>> Chain FORWARD (policy ACCEPT)
>>>>> target prot opt source destination
>>>>> SOLUSVM_TRAFFIC_IN all -- anywhere anywhere
>>>>> SOLUSVM_TRAFFIC_OUT all -- anywhere anywhere
>>>>>
>>>>> Chain OUTPUT (policy ACCEPT)
>>>>> target prot opt source destination
>>>>>
>>>>> Chain SOLUSVM_TRAFFIC_IN (1 references)
>>>>> target prot opt source destination
>>>>> all -- anywhere 192.168.1.14
>>>>>
>>>>> Chain SOLUSVM_TRAFFIC_OUT (1 references)
>>>>> target prot opt source destination
>>>>> all -- 192.168.1.14 anywhere
>>>>>
>>>>> kind regards
>>>>> Stefan Schmitz
>>>>>
>>>>>
>>>>
>
More information about the Users
mailing list