[ClusterLabs] Antw: Re: Antw: why is node fenced ?

[Ulrich Windl]

>>> "Lentes, Bernd" <bernd.lentes at helmholtz-muenchen.de> schrieb am 23.05.2019
um
15:01 in Nachricht
<1029244418.9784641.1558616472505.JavaMail.zimbra at helmholtz-muenchen.de>:

> 
> ‑‑‑‑‑ On May 20, 2019, at 8:28 AM, Ulrich Windl
Ulrich.Windl at rz.uni‑regensburg.de 
> wrote:
> 
>>>>> "Lentes, Bernd" <bernd.lentes at helmholtz‑muenchen.de> schrieb am
16.05.2019
>> um
>> 17:10 in Nachricht
>> <1151882511.6631123.1558019430655.JavaMail.zimbra at helmholtz‑muenchen.de>:
>>> Hi,
>>> 
>>> my HA‑Cluster with two nodes fenced one on 14th of may.
>>> ha‑idg‑1 has been the DC, ha‑idg‑2 was fenced.
>>> It happened around 11:30 am.
>>> The log from the fenced one isn't really informative:
>>> 
>>> ==================================
>>> 2019‑05‑14T11:22:09.948980+02:00 ha‑idg‑2 liblogging‑stdlog: ‑‑ MARK ‑‑
>>> 2019‑05‑14T11:28:21.548898+02:00 ha‑idg‑2 sshd[14269]: Accepted
>>> keyboard‑interactive/pam for root from 10.35.34.70 port 59449 ssh2
>>> 2019‑05‑14T11:28:21.550602+02:00 ha‑idg‑2 sshd[14269]:
>>> pam_unix(sshd:session): session opened for user root by (uid=0)
>>> 2019‑05‑14T11:28:21.554640+02:00 ha‑idg‑2 systemd‑logind[2798]: New
session
>> 
>>> 15385 of user root.
>>> 2019‑05‑14T11:28:21.555067+02:00 ha‑idg‑2 systemd[1]: Started Session
15385
>> 
>>> of user root.
>>> 
>>> 2019‑05‑14T11:44:07.664785+02:00 ha‑idg‑2 systemd[1]: systemd 228 running
in
>> 
>>> system mode. (+PAM ‑AUDIT +SELINUX ‑IMA +APPARMOR ‑SMACK +SYSVINIT +UTMP
>>> +LIBCRYPTSETUP +GC   Neustart !!!
>>> RYPT ‑GNUTLS +ACL +XZ ‑LZ4 +SECCOMP +BLKID ‑ELFUTILS +KMOD ‑IDN)
>>> 2019‑05‑14T11:44:07.664902+02:00 ha‑idg‑2 kernel: [    0.000000] Linux
>>> version 4.12.14‑95.13‑default (geeko at buildhost) (gcc version 4.8.5 (SUSE
>>> Linux) ) #1 SMP Fri Mar
>>> 22 06:04:58 UTC 2019 (c01bf34)
>>> 2019‑05‑14T11:44:07.665492+02:00 ha‑idg‑2 systemd[1]: Detected
architecture
>> 
>>> x86‑64.
>>> 2019‑05‑14T11:44:07.665510+02:00 ha‑idg‑2 kernel: [    0.000000] Command
>>> line: BOOT_IMAGE=/boot/vmlinuz‑4.12.14‑95.13‑default
>>> root=/dev/mapper/vg_local‑lv_root resume=/
>>> dev/disk/by‑uuid/2849c504‑2e45‑4ec8‑bbf8‑724cf358ee25 splash=verbose
>>> showopts
>>> 2019‑05‑14T11:44:07.665510+02:00 ha‑idg‑2 systemd[1]: Set hostname to
>>><ha‑idg‑2>.
>>> =================================
> 
>>> 
>>> One network interface is gone for a short period. But it's in a bonding
>>> device (round‑robin),
>>> so the connection shouldn't be lost. Both nodes are connected directly,
>>> there is no switch in between.
>> 
>> I think you misunderstood: a round‑robin bonding device is not fault‑safe
>> IMHO, but it depends a lot on your cabling details. Also you did not show 
> the
>> logs on the other nodes.
> 
> /usr/src/linux/Documentation/networking/bonding.txt says:
> " balance‑rr or 0: Round‑robin policy: Transmit packets in sequential
>                 order from the first available slave through the
>                 last.  This mode provides load balancing >> and fault
>                 tolerance. <<

We had that, and when one of two cables had failed at side A, side B would
still use both, so A would receive only every 2nd packet or so....

> Nevertheless i think i will switch to active/backup.
> 
> I showed up /var/log/messages from the fenced note above.
> Or do you mean /var/log/pacemaker.log from the fenced one ?
> Isn't that the same as the one from the DC ?
> 
>  
>>> I manually (ifconfig eth3 down) stopped the interface afterwards several
>>> times ... nothing happened.
>>> The same with the second Interface (eth2).
> 
> That means i also stopped several time the second interface, also with no 
> fencing of the node.
> 
> Bernd
>  
> 
> Helmholtz Zentrum Muenchen
> Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH)
> Ingolstaedter Landstr. 1
> 85764 Neuherberg
> www.helmholtz‑muenchen.de 
> Stellv. Aufsichtsratsvorsitzender: MinDirig. Dr. Manfred Wolter
> Geschaeftsfuehrung: Prof. Dr. med. Dr. h.c. Matthias Tschoep, Heinrich 
> Bassler, Kerstin Guenther
> Registergericht: Amtsgericht Muenchen HRB 6466
> USt‑IdNr: DE 129521671
> 
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users 
> 
> ClusterLabs home: https://www.clusterlabs.org/