[Pacemaker] Multinode cluster question

[Attila Megyeri]

-----Original Message-----
From: Andreas Kurz [mailto:andreas at hastexo.com] 
Sent: 2011. november 10. 11:03
To: pacemaker at oss.clusterlabs.org
Subject: Re: [Pacemaker] Multinode cluster question

On 11/09/2011 12:04 AM, Attila Megyeri wrote:
> -----Original Message-----
> From: Attila Megyeri [mailto:amegyeri at minerva-soft.com]
> Sent: 2011. november 8. 16:13
> To: The Pacemaker cluster resource manager
> Subject: Re: [Pacemaker] Multinode cluster question
> 
> -----Original Message-----
> From: Andreas Kurz [mailto:andreas at hastexo.com]
> Sent: 2011. november 8. 15:27
> To: pacemaker at oss.clusterlabs.org
> Subject: Re: [Pacemaker] Multinode cluster question
> 
> On 11/08/2011 03:14 PM, Attila Megyeri wrote:
>>
>> -----Original Message-----
>> From: Andreas Kurz [mailto:andreas at hastexo.com]
>> Sent: 2011. november 8. 15:03
>> To: pacemaker at oss.clusterlabs.org
>> Subject: Re: [Pacemaker] Multinode cluster question
>>
>> On 11/08/2011 02:50 PM, Attila Megyeri wrote:
>>> -----Original Message-----
>>> From: Andreas Kurz [mailto:andreas at hastexo.com]
>>> Sent: 2011. november 8. 14:42
>>> To: pacemaker at oss.clusterlabs.org
>>> Subject: Re: [Pacemaker] Multinode cluster question
>>>
>>> On 11/08/2011 12:02 PM, Attila Megyeri wrote:
>>>> Hi All,
>>>>
>>>>  
>>>>
>>>> I need some help/guidance, on how to make sure that certain 
>>>> resources (running in virtual cluster nodes) are run on the same physical server.
>>>>
>>>>  
>>>>
>>>> The setup:
>>>>
>>>>  
>>>>
>>>> I have a cluster made of two physical nodes, that I am willing to 
>>>> use for HA purposes (no LB for the time being).
>>>>
>>>> I have a failover IPfrom the provider, that is controlled using a 
>>>> resource agent from one pair of the virtual machines(web1 and 
>>>> web2), and the IP is assigned always to one of the physical servers.
>>>>
>>>> On the physical server I use iptables pre/postrouting to direct the 
>>>> traffic to the appropriate virtual node.The routing points to the 
>>>> web VIP, and red5 VIP.
>>>>
>>>>  
>>>>
>>>> On the physical servers I have 3-3 virtual servers, that host the 
>>>> specific roles of the solution, e.g. db1 db2, web1 web2, red5_1 red5_2.
>>>>
>>>> The virtual servers use the default gateway of their own physical 
>>>> server to talk to the outside world.
>>>>
>>>>  
>>>>
>>>> My first idea was to create 3 independent two-node clusters. Db 
>>>> cluster, web cluster, red5 cluster.
>>>>
>>>> The db cluster is a M/S psql, with a virtual IP.
>>>>
>>>> The web cluster is an apache2 cluster, cloned on two virtual 
>>>> servers, with a failover IP RA (if node1 on phy1 fails, failover Ip 
>>>> is redirected to phy2 and vice versa).
>>>>
>>>> Red5 is a red5 cluster running on two instances, with a virtual IP 
>>>> (internal).
>>>>
>>>>  
>>>>
>>>> This is where it gets interesting - because of the default gateway.
>>>>
>>>> The db cluster is accessed from the intranet only - no worries here.
>>>>
>>>>  
>>>>
>>>> Red5 is different - but it needs further explanation.
>>>>
>>>> Let's assume that all roles (db master, web, red5) are running on 
>>>> phisical server  1.
>>>>
>>>> Web1 fails for some reason. Web2 role will become active, and the 
>>>> external failover IP will point from now on to physical node2.  The 
>>>> iptables script still points to the same VIP address, but it now 
>>>> runs on a different node. No issue shere, as Web2  gets its traffic 
>>>> properly, as it KNOWs that it is running on node2 now.
>>>>
>>>>  
>>>>
>>>> The issue is with Red5.
>>>>
>>>> Red5 runs on node1, and uses default gw on node1. [it does not know 
>>>> that the external failover IP no longer points to node1].
>>>>
>>>> When a request is received on the failover IP (now ph node2), 
>>>> iptables redirects it to red5's VIP. Red5, running on node1 gets 
>>>> this request, but does not know that it shall be routed through node2!
>>>>
>>>> As such, the replies, will be routed through ph node1 - as it is 
>>>> the default gw. This is definitively not the right approach.
>>>>
>>>>  
>>>>
>>>> The actual question is:
>>>>
>>>> -          Should I treat all nodes inside the same cluster (db1, db2,
>>>> web1, web2, red1, red2) - and this way I could possibly detect that 
>>>> failover IP has changed and I should "do something" with red5?
>>>>
>>>> -          "Do something" could mean for me one of the following:
>>>>
>>>> o   If "web" VIP is running on physical node 2 (on node "web2"), then
>>>> move "red" VIP to physical node2 (to node "red2")
>>>>
>>>> o   Alternatively, only change the default gateway for red1, to use
>>>> "node2" as the default gateway?
>>>
>>> Why not doing all traffic between the VMs via an internal net only ...
>>> e.g. a direct connnected nic between hosts, bridged and all VMs connected to it?
>>>
>>>
>>>
>>> I'm not sure I got you right.
>>> All VMs are connected and bridged. All VMs can use any of the physical hosts as their default gateway. And there is no issue with internal traffic.
>>> The questions is what to do when a resource runs on node1 (default GW is host1), but the external failover IP points to host2, and host2 routes the packets to VIP of the resource. The outgoing packet will leave the internal network on host1, and it entered on host2.
>>
>> If they are "all bridged and connected" why would you have a routing problem? They are all in the same net, no routing involved ... only the arp caches need to be updated after an IP failover ... this is handled e.g. in the IPAddr2 RA of Pacemaker.
>>
>>
>> The problem is that red1 is not failed over when web1 fails, e.g. it still runs on node1.
>> Or were you thinking about a VIP for the default gateway as well?
> 
> Maybe I misinterpreted your description, so pleas correct me if my assumptions are wrong ... but if all VMs are in the same subnet, no routing is involved so you don't need to take care about the default gateway ... it is not used when there is a direct link into this network.
> 
> 
> The VMs ARE inside the same subnet, but clients are accessing the website from the public Internet.
> As we have two hosts pcs, with two public internet connections and a third failover IP, depending on where the failover ip points (host1 or host2) we are getting queries through different routes.
> In nutshell - two gateways to the public internet. And the VM does not know which one to use.
> 
> 
> 
> I think I know what you might be misinterpreting:
> 
> This is not a network setup, where we have a router/firewall connected to the Internet and the rest of the servers are in the intranet. The host servers themselves are connected to the public internet and they provide the routing/firewall function as well. As such, they represent two gateways to the public net. I hope this clears up the situation a bit.
> 

I see. You could do a SNAT using physical node IP when redirecting to your Red5 to be sure it always is routed through the incoming host ...
or always route Red5 through the active webserver, though that would require an extra VIP.


Well, SNAT is not really an option if I want to track the users for statistical purposes. I will look into the routing option as well.

Thanks,
Attila