[Pacemaker] Corosync using unicast instead of multicast

Mon Nov 8 07:50:24 EST 2010

Hi,

Steven Dake wrote:
> On 11/05/2010 01:30 AM, Dan Frincu wrote:
>> Hi,
>>
>> Alan Jones wrote:
>>> This question should be on the openais list, however, I happen to know
>>> the answer.
>>> To get up and running quickly you can configure broadcast with the
>>> version you have.
>>>
>> I've done that already, however I was a little concerned as to what
>> Steven Dake said on the openais mailing list about using broadcast
>> "Broadcast and redundant ring probably don't work to well together.".
>>
>> I've also done some testing and saw that the broadcast address used is
>> 255.255.255.255, regardless of what the bindnetaddr network address is,
>> and quite frankly, I was hoping to see a directed broadcast address.
>> This wasn't the case, therefore I wonder whether this was the issue that
>> Steven was referring to, because by using the 255.255.255.255 as a
>> broadcast address, there is the slight chance that some application
>> running in the same network might send a broadcast packet using the same
>
> This can happen with multicast or unicast modes as well.  If a third 
> party application communicates on the multicast/port combo or unicast 
> port of a cluster node, there is conflict.
>
> With encryption, corosync encrypts and authenticates all packets, 
> ignoring packets without a proper signature.  The signatures are 
> difficult to spoof.  Without encryption, bad things happen in this 
> condition.
>
> For more details, read "SECURITY" file in our source distribution.
>
OK, I read the SECURITY file, a lot of overhead is added, I understand 
the reasons why it does it this way, not going to go into the details 
right now. Basically enabling encryption ensures that any traffic going 
between the nodes is both encrypted and authenticated, so rogue messages 
that happen to reach the exact network socket will be discarded. I'll 
come back to this a little bit later.

Then again, I have this sentence in my head that I can't seem to get rid 
of "Broadcast and redundant ring probably don't work to well together, 
broadcast and redundant ring probably don't work to well together...." 
and also I read "OpenAIS now provides broadcast network communication in 
addition to multicast. This functionality is considered Technology 
Preview for standalone usage of OpenAIS", therefore I'm a little bit 
more concerned.

Can you shed some light on this please? Two questions:

1) What do you mean by "Broadcast and redundant ring probably don't work 
to well together"?

2) Is using Corosync's broadcast feature instead of multicast stable 
enough to be used in production systems?

Thank you in advance.

Best regards,

Dan
>> port as configured on the cluster. How would the cluster react to that,
>> would it ignore the packet, would it wreak havoc?
>>
>> Regards,
>>
>> Dan
>>
>> That's my main concern right now.
>>> Corosync can distinguish separate clusters with the multicast address
>>> and port that become payload to the messages.
>>> The patch you referred to can be applied to the top of tree for
>>> corosync or you can wait for a new release 1.3.0 planned for the end
>>> of November.
>>> Alan
>>>
>>> On Thu, Nov 4, 2010 at 1:02 AM, Dan Frincu<dfrincu at streamwide.ro>  
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I'm having an issue with a setup using the following:
>>>> cluster-glue-1.0.6-1.6.el5.x86_64.rpm
>>>> cluster-glue-libs-1.0.6-1.6.el5.x86_64.rpm
>>>> corosync-1.2.7-1.1.el5.x86_64.rpm
>>>> corosynclib-1.2.7-1.1.el5.x86_64.rpm
>>>> drbd83-8.3.2-6.el5_3.x86_64.rpm
>>>> kmod-drbd83-8.3.2-6.el5_3.x86_64.rpm
>>>> openais-1.1.3-1.6.el5.x86_64.rpm
>>>> openaislib-1.1.3-1.6.el5.x86_64.rpm
>>>> pacemaker-1.0.9.1-1.el5.x86_64.rpm
>>>> pacemaker-libs-1.0.9.1-1.el5.x86_64.rpm
>>>> resource-agents-1.0.3-2.el5.x86_64.rpm
>>>>
>>>> This is a two-node HA cluster, with the nodes interconnected via 
>>>> bonded
>>>> interfaces through the switch. The issue is that I have no control 
>>>> of the
>>>> switch itself, can't do anything about that, and from what I 
>>>> understand the
>>>> environment doesn't allow enabling multicast on the switch. In this
>>>> situation, how can I have the setup functional (with redundant rings,
>>>> rrp_mode: active) without using multicast.
>>>>
>>>> I've seen that individual network sockets are formed between nodes, 
>>>> unicast
>>>> sockets, as well as the multicast sockets. I'm interested in 
>>>> knowing how
>>>> will the lack of multicast affect the redundant rings, connectivity,
>>>> failover, etc.
>>>>
>>>> I've also seen this page
>>>> https://lists.linux-foundation.org/pipermail/openais/2010-October/015271.html 
>>>>
>>>> And here it states using UDPU transport mode avoids using multicast or
>>>> broadcast, but it's a patch, is this integrated in any of the newer 
>>>> versions
>>>> of corosync?
>>>>
>>>> Thank you in advance.
>>>>
>>>> Regards,
>>>>
>>>> Dan
>>>>
>>>> -- 
>>>> Dan FRINCU
>>>> Systems Engineer
>>>> CCNA, RHCE
>>>> Streamwide Romania
>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list:Pacemaker at oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home:http://www.clusterlabs.org
>>>> Getting 
>>>> started:http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs:
>>>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker 
>>>>
>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Pacemaker mailing list:Pacemaker at oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home:http://www.clusterlabs.org
>>> Getting started:http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs:http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker 
>>>
>>>
>>
>> -- 
>> Dan FRINCU
>> Systems Engineer
>> CCNA, RHCE
>> Streamwide Romania
>>
>>
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: 
>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker 
>>
>

-- 
Dan FRINCU
Systems Engineer
CCNA, RHCE
Streamwide Romania