[ClusterLabs] large cluster with corosync
Arne Jansen
lists at die-jansens.de
Wed Jan 4 05:33:21 EST 2017
On 04.01.2017 11:25, Kristoffer Grönlund wrote:
> Arne Jansen <lists at die-jansens.de> writes:
>
>> Hi,
>>
>> I've built corosync for solaris and am trying to build a largish
>> cluster. I started corosync with default configuration on an
>> increasing number of nodes, one by one. At around 70 nodes the
>> cluster breaks down. Below is an excerpt from the logfile on the
>> first node.
>> When the cluster breaks down corosync seems to completely flood
>> the network. It doesn't recover by itself, I have to stop all nodes.
>> Things get worse if I start corosync on multiple nodes at once.
>> In this case it already breaks down around 40 nodes.
>>
>> Is it supposed to work with such a setup? Does it just need tuning?
>> My goal is have a cluster of several hundred nodes.
>
> Hi,
>
> No, corosync has a limit to how many nodes the cluster can
> contain and still function properly. According to SUSE, the limit is 32
> nodes, according to Red Hat the limit is 16 (those are the limits for
> enterprise support as far as I know) - but even having that number
> of nodes will probably require some tweaking of timeouts in
> corosync.conf, depending on what kind of network you have.
>
> If I recall correctly, there are plans for overcoming this limit in the
> future but right now that's the situation.
>
> There is the pacemaker_remote project which allows for adding
> additional, non-corosync nodes to a cluster which can run resources but
> don't participate in quorum and rely on at least some of the core
> cluster nodes still being available. Using pacemaker_remote could enable
> a cluster of several hundred nodes.
>
> My question would be, why do you need so many nodes in a high
> availaibility cluster?
At least those limits doesn't seem to get enforced, as a 64 node cluster
seems to work, although a bit shaky.
I want to use corosync + dlm to get a distributed lock manager to have
a uniform locking service throughout the platform as a basis for many
services.
Thanks,
Arne
>
> Cheers,
> Kristoffer
>
>>
>> Thanks,
>> Arne
>>
>> Jan 04 10:24:49 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:25:00 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:616) was formed. Members joined: 168101089
>> Jan 04 10:25:00 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:25:11 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:620) was formed. Members joined: 168101090
>> Jan 04 10:25:11 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:25:21 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:624) was formed. Members joined: 168101091
>> Jan 04 10:25:21 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:25:32 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:628) was formed. Members joined: 168101092
>> Jan 04 10:25:32 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:26:09 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:632) was formed. Members joined: 168101141
>> Jan 04 10:26:10 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:26:20 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:636) was formed. Members joined: 168101142
>> Jan 04 10:26:20 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:26:31 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:640) was formed. Members joined: 168101143
>> Jan 04 10:26:31 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:26:42 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:644) was formed. Members joined: 168101144
>> Jan 04 10:26:42 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:26:52 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:648) was formed. Members joined: 168101145
>> Jan 04 10:26:52 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:27:03 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:652) was formed. Members joined: 168101146
>> Jan 04 10:27:03 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:27:14 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:656) was formed. Members joined: 168101147
>> Jan 04 10:27:14 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:27:25 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:660) was formed. Members joined: 168101148
>> Jan 04 10:27:25 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:27:35 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:664) was formed. Members joined: 168101161
>> Jan 04 10:27:35 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:27:46 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:668) was formed. Members joined: 168101162
>> Jan 04 10:27:46 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:27:57 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:672) was formed. Members joined: 168101163
>> Jan 04 10:27:57 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:28:07 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:676) was formed. Members joined: 168101164
>> Jan 04 10:28:07 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:28:18 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:680) was formed. Members joined: 168101165
>> Jan 04 10:28:18 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:28:28 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:684) was formed. Members joined: 168101166
>> Jan 04 10:28:28 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:28:40 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:688) was formed. Members joined: 168101167
>> Jan 04 10:28:40 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:28:50 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:692) was formed. Members joined: 168101168
>> Jan 04 10:28:50 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:29:00 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:696) was formed. Members joined: 168101181
>> Jan 04 10:29:00 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:29:14 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:704) was formed. Members joined: 168100966 168100967
>> 168100968 168100969 168100970 168100971 168100972 168100985 168100986
>> 168100987 168100988 168100989 168100990 168100991 168100992 168101005
>> 168101006 168101007 168101008 168101009 168101010 168101011 168101012
>> 168101025 168101026 168101027 168101028 168101029 168101030 168101031
>> 168101032 168101045 168101046 168101047 168101048 168101049 168101050
>> 168101051 168101052 168
>> Jan 04 10:29:14 [4915] reniar corosync notice [TOTEM ] Failed to
>> receive the leave message. failed: 168100966 168100967 168100968
>> 168100969 168100970 168100971 168100972 168100985 168100986 168100987
>> 168100988 168100989 168100990 168100991 168100992 168101005 168101006
>> 168101007 168101008 168101009 168101010 168101011 168101012 168101025
>> 168101026 168101027 168101028 168101029 168101030 168101031 168101032
>> 168101045 168101046 168101047 168101048 168101049 168101050 168101051
>> 168101052 168101065 168101066
>> Jan 04 10:29:15 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:716) was formed. Members joined: 168100966 168100967
>> 168100968 168100969 168100970 168100971 168100972 168100985 168100986
>> 168100987 168100988 168100989 168100990 168100991 168100992 168101005
>> 168101006 168101007 168101008 168101009 168101010 168101011 168101012
>> 168101025 168101026 168101027 168101028 168101029 168101030 168101031
>> 168101032 168101045 168101046 168101047 168101048 168101049 168101050
>> 168101051 168101052 168
>> Jan 04 10:29:15 [4915] reniar corosync notice [TOTEM ] Failed to
>> receive the leave message. failed: 168100966 168100967 168100968
>> 168100969 168100970 168100971 168100972 168100985 168100986 168100987
>> 168100988 168100989 168100990 168100991 168100992 168101005 168101006
>> 168101007 168101008 168101009 168101010 168101011 168101012 168101025
>> 168101026 168101027 168101028 168101029 168101030 168101031 168101032
>> 168101045 168101046 168101047 168101048 168101049 168101050 168101051
>> 168101052 168101066 168101067
>> Jan 04 10:29:16 [4915] reniar corosync notice [TOTEM ] Retransmit List:
>> 285
>> Jan 04 10:29:16 [4915] reniar corosync notice [TOTEM ] Retransmit List:
>> 4db 4dc 4dd 4e1
>> Jan 04 10:29:16 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:20 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:732) was formed. Members left: 168100966 168100967 168100968
>> 168100969 168100970 168100971 168100972 168100985 168100986 168100987
>> 168100988 168100989 168100990 168100991 168100992 168101005 168101006
>> 168101007 168101008 168101009 168101010 168101011 168101012 168101025
>> 168101026 168101027 168101028 168101029 168101030 168101031 168101032
>> 168101045 168101046 168101047 168101048 168101049 168101050 168101051
>> 168101052 16810
>> Jan 04 10:29:20 [4915] reniar corosync notice [TOTEM ] Failed to
>> receive the leave message. failed: 168100966 168100967 168100968
>> 168100969 168100970 168100971 168100972 168100985 168100986 168100987
>> 168100988 168100989 168100990 168100991 168100992 168101005 168101006
>> 168101007 168101008 168101009 168101010 168101011 168101012 168101025
>> 168101026 168101027 168101028 168101029 168101030 168101031 168101032
>> 168101045 168101046 168101047 168101048 168101049 168101050 168101051
>> 168101052 168101066 168101067
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:736) was formed. Members
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:21 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:25 [4915] reniar corosync notice [TOTEM ] Process pause
>> detected for 503 ms, flushing membership messages.
>> Jan 04 10:29:27 [4915] reniar corosync notice [TOTEM ] Process pause
>> detected for 1277 ms, flushing membership messages.
>> Jan 04 10:29:27 [4915] reniar corosync warning [MAIN ] Corosync main
>> process was not scheduled for 1345.9542 ms (threshold is 800.0000 ms).
>> Consider token timeout increase.
>> Jan 04 10:29:29 [4915] reniar corosync notice [TOTEM ] Process pause
>> detected for 607 ms, flushing membership messages.
>> Jan 04 10:29:42 [4915] reniar corosync warning [MAIN ] Corosync main
>> process was not scheduled for 12273.4664 ms (threshold is 800.0000 ms).
>> Consider token timeout increase.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:916) was formed. Members
>> Jan 04 10:29:42 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:42 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync notice [TOTEM ] Process pause
>> detected for 12562 ms, flushing membership messages.
>> Jan 04 10:29:55 [4915] reniar corosync warning [MAIN ] Corosync main
>> process was not scheduled for 12636.5143 ms (threshold is 800.0000 ms).
>> Consider token timeout increase.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync notice [TOTEM ] A new membership
>> (10.5.4.101:964) was formed. Members
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync notice [MAIN ] Completed
>> service synchronization, ready to provide service.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:55 [4915] reniar corosync warning [TOTEM ] JOIN or LEAVE
>> message was thrown away during flush operation.
>> Jan 04 10:29:59 [4915] reniar corosync warning [MAIN ] Corosync main
>> process was not scheduled for 1465.7160 ms (threshold is 800.0000 ms).
>> Consider token timeout increase.
>>
>> _______________________________________________
>> Users mailing list: Users at clusterlabs.org
>> http://lists.clusterlabs.org/mailman/listinfo/users
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>>
>
More information about the Users
mailing list