[ClusterLabs] Totem is unable to form a cluster because of an operating system or network fault

[dinor geler]

Hi ,
Am trying to configure my sql on ubuntu according to this article :
https://azure.microsoft.com/en-in/documentation/articles/virtual-machines-linux-classic-mysql-cluster/

two node cluster


looking on corosync log :


Apr 12 11:01:09 corosync [TOTEM ] Totem is unable to form a cluster because
of an operating system or network fault. The most common cause of this
message is that the local firewall is configured improperly.
Apr 12 11:01:11 corosync [TOTEM ] Totem is unable to form a cluster because
of an operating system or network fault. The most common cause of this
message is that the local firewall is configured improperly.
Apr 12 11:01:13 corosync [TOTEM ] Totem is unable to form a cluster because
of an operating system or network fault. The most common cause of this
message is that the local firewall is configured improperly.
Apr 12 11:01:16 corosync [TOTEM ] Totem is unable to form a cluster because
of an operating system or network fault. The most common cause of this
message is that the local firewall is configured improperly.
Apr 12 11:01:18 corosync [TOTEM ] Totem is unable to form a cluster because
of an operating system or network fault. The most common cause of this
message is that the local firewall is configured improperly.
Apr 12 11:01:20 corosync [TOTEM ] Totem is unable to form a cluster because
of an operating system or network fault. The most common cause of this
message is that the local firewall is configured improperly.
Apr 12 11:01:22 corosync [TOTEM ] Totem is unable to form a cluster because
of an operating system or network fault. The most common cause of this
message is that the local firewall is configured improperly.
Apr 12 11:01:24 corosync [TOTEM ] Totem is unable to form a cluster because
of an operating system or network fault. The most common cause of this
message is that the local firewall is configured improperly.
Apr 12 11:01:27 corosync [TOTEM ] Totem is unable to form a cluster because
of an operating system or network fault. The most common cause of this
message is that the local firewall is configured improperly.
Apr 12 11:01:29 corosync [TOTEM ] Totem is unable to form a cluster because
of an operating system or network fault. The most common cause of this
message is that the local firewall is configured improperly.
Apr 12 11:01:31 corosync [TOTEM ] Totem is unable to form a cluster because
of an operating system or network fault. The most common cause of this
message is that the local firewall is configured improperly.



totem {
  version: 2
  crypto_cipher: none
  crypto_hash: none
  interface {
    ringnumber: 0
    bindnetaddr: 10.1.0.0
    mcastport: 5405
    ttl: 1
  }
  transport: udpu
}
logging {
  fileline: off
  to_logfile: yes
  to_syslog: yes
  logfile: /var/log/corosync/corosync.log
  debug: off
  timestamp: on
  logger_subsys {
    subsys: QUORUM
    debug: off
    }
  }
nodelist {
  node {
    ring0_addr: 10.1.0.6
    nodeid: 1
  }
  node {
    ring0_addr: 10.1.0.7
    nodeid: 2
  }
}
quorum {
  provider: corosync_votequorum
}


If I initiate a tcpdump on node 2 and start either a netcat or nmap I see
packet arrives to destination host for port 5405 UDP traffic



I do see Corosync listening on the IP/PORT





root at node-2:/home/dinor# netstat -an | grep -i 5405

udp        0      0 10.1.0.7:5405           0.0.0.0:*






root at node-1:/home/dinor# netstat -an | grep -i 5405

udp        0      0 10.1.0.6:5405           0.0.0.0:*





On node 1 I start a netcat to port 5405 via udp



netcat -D -4 -u 10.1.0.7 5405



In here you type some text and hit enter



On node 1 tcpdump we see data sent to IP 10.1.0.7



root at node-1:/var/log/corosync# tcpdump -n udp port 5405

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

10:08:24.484533 IP 10.1.0.6.44299 > 10.1.0.7.5405: UDP, length 26







On node 2 tcpdump I see the data arrive



root at node-2:/var/log/corosync# tcpdump -n udp port 5405

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

10:08:24.484892 IP 10.1.0.6.44299 > 10.1.0.7.5405: UDP, length 26





Tested also sending UDP packets from node 2 – all ok.



So connectivity seems to be ok.



Port scanner also shows the port as Open





root at node-1:/home/dinor# nmap -sUV 10.1.0.7 -p 5402-5405



Starting Nmap 5.21 ( http://nmap.org ) at 2016-04-12 10:31 UTC

Nmap scan report for node-2 (10.1.0.7)

Host is up (0.00060s latency).

PORT     STATE         SERVICE VERSION

5402/udp closed        unknown

5403/udp closed        unknown

5404/udp closed        unknown

*5405/udp open|filtered unknown*

MAC Address: 12:34:56:78:9A:BC (Unknown)



Service detection performed. Please report any incorrect results at
http://nmap.org/submit/ .

Nmap done: 1 IP address (1 host up) scanned in 79.07 seconds





There is no FW and no selinux enabled


help appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.clusterlabs.org/pipermail/users/attachments/20160412/184f5189/attachment-0002.html>