[ClusterLabs] Pacemaker ocf:heartbeat:IPaddr on different subnets

[Rowan @ Jetboy]

"It looks like you have a networking issue rather than a clustering issue."

That's proven to be the case. I'm hosting ESXi with online.net, who 
provide additional 'failover' IPs that can be assigned to my hosted VMs. 
I have three; 200.xx.xxx.9 and 200.xx.xxx.10 which I'm using for the two 
Ubuntu VMs listed, and the third, 200.zzz.z.162, which I'm trying to use 
as an ocf_heartbeat_IPaddr resource agent. Critically, online.net 
require that you match assign each IP a MAC address that matches an 
interface on a VM. I did this for the first two IPs (using different MAC 
addresses) but not for the third. When I tried using one of the existing 
MAC addresses with the third IP, I immediately got connectivity.

So now, I have all three IPs and both VM interfaces sharing the same MAC 
address. This strikes me as a *really bad idea*, but, it actually seems 
to be working, without creating any packet errors or problems with 
connectivity or the HA cluster... at least none that I've spotted so far.


On 22/06/2015 14:56, Ken Gaillot wrote:
> On 06/20/2015 01:59 AM, Rowan @ Jetboy wrote:
>> [Ubuntu 14 LTS on VMware ESXi 6; Corosync; Pacemaker 1.1.10]
>>
>> I'm trying to add a Pacemaker virtual IP address; with it, the gateway,
>> and the two VMs it serves on different subnets. I've only done this
>> before with all IPs on the same subnet, and I need some help.
>>
>> I have two VMs on 200.xx.xxx.9 and 200.xx.xxx.10 with the below in
>> `/etc/network/interfaces`
>>
>>      auto eth0
>>      iface eth0 inet static
>>          address 200.xx.xxx.9
>>          gateway 200.xx.xxx.9
>>          netmask 255.255.255.255
>>
>>          post-up route add yy.yyy.yyy.1 dev eth0
>>          post-up route add default gw yy.yyy.yyy.1
> It looks like you have a networking issue rather than a clustering
> issue. I'm not sure what the requirements/goals are for having the
> different subnets, but the default gateway should be on a directly
> connected IP network (not over a route), and the gateway itself must
> know where to route each IP network.
>
> Are all these subnets on the same (physical or VLAN) Ethernet network?
> Or are there multiple interfaces involved?
>
>> and
>>
>>      auto eth0
>>      iface eth0 inet static
>>          address 200.xx.xxx.10
>>          gateway 200.xx.xxx.10
>>          netmask 255.255.255.255
>>
>>          post-up route add yy.yyy.yyy.1 dev eth0
>>          post-up route add default gw yy.yyy.yyy.1
>>
>> They're both showing up in Pacemaker, and seemingly communicating OK.
>> The bindnetaddr parameters in the two `/etc/corosync/corosync.conf`
>> files are:
>>
>>      bindnetaddr: address 200.xx.xxx.9
>>
>> and
>>
>>      bindnetaddr: address 200.xx.xxx.10
>>
>> respectively.
>>
>> If everything was on the same subnet, I'd expect to add the virtual IP
>> with something like:
>>
>>      sudo crm configure primitive eth0_virtual ocf:heartbeat:IPaddr
>> params ip="200.zzz.z.162" nic="eth0" cidr_netmask="24"
>> broadcast="200.zzz.z.255" op monitor interval="10s" timeout="20s"
>>
>> and while this shows up as resource in crm_mon, it isn't allowing me to
>> access one of the VMs via the virtual IP. Clearly there's more to it,
>> but what?