[Pacemaker] pacemaker/corosync: a resource is started on 2 nodes

[Michael Schwartzkopff]

Am Mittwoch, 28. Januar 2015, 14:20:51 schrieb Sergey Arlashin:
> Hi!
> 
> I have a small corosync/pacemaker based cluster which consists of 4 nodes. 2
> nodes are in standby mode, another 2 actually handle all the resources.
> 
> corosync  ver. 1.4.7-1.
> pacemaker  ver  1.1.11.
> os: ubuntu 12.04.
> 
> Inside our production environment which has a plenty of free ram,cpu etc
> everything is working well. When I switch one node off all the resources
> move to another without any problems. And vice versa. That's what I need :)
> 
> Our staging environment has rather weak hardware (that's ok - it's just
> staging :) ) and is rather busy. Sometimes it even doesn't have enough cpu
> or disk speed to be stable. When that happens some of cluster resources
> fail (which I consider to be normal), but also I can see the following crm
> output:
> 
> Node db-node1: standby
> Node db-node2: standby
> Online: [ lb-node1 lb-node2 ]
> 
>  Pgpool2	(ocf::heartbeat:pgpool):	FAILED (unmanaged) [ lb-node2 lb-node1 ]
>  Resource Group: IPGroup
>      FailoverIP1	(ocf::heartbeat:IPaddr2):	Started [ lb-node2 lb-node1 ]
> 
> As you can see the resource ocf::heartbeat:IPaddr2 is started on both nodes
> ( lb-node2 and lb-node1 ). But I can't figure out how than could happen.

Your config does not allow this, but since your HW is slow pacemaker runs into 
timeouts and corosync conneciton problems. You could debug the problem be 
tracing the event in the logs. With the command crm_mon -1rtf you find the time 
of the failure. Search around that time in the logs.

If the communication in the cluster does not work, pacemaker sometimes behaves 
verry odd.

Mit freundlichen Grüßen,

Michael Schwartzkopff

-- 
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044
Franziskanerstraße 15, 81669 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein