[Pacemaker] corosync vs. pacemaker 1.1

[Kiss Bence]

Hi,

I am newbie to the clustering and I am trying to build a two node 
active/passive cluster based upon the documentation: 
http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf

My systems are Fedora 14, uptodate. After forming the cluster as wrote, 
I started to test it. (resources: drbd-> lvm-> fs ->group of services) 
Resources moved around, nodes rebooted and killed (first I tried it in 
virtual environment then also on real machines).

After some events the two nodes ended up in a kind of state of 
split-brain. The crm_mon showed me that the other node is offline at 
both nodes although the drbd subsystem showed everything in sync and 
working. The network was not the issue (ping, tcp and udp communications 
were fine). Nothing changed from the network view.

At first the rejoining took place quiet well, but some more events after 
it took longer and after more event it didn't. The network dump showed 
me the multicast packets still coming and going. At corosync (crm_node 
-l) the other node didn't appeared both on them. After trying 
configuring the cib logs was full of messages like "<the other node>: 
not in our membership".

I tried to erase the config (crm configure erase, cibadmin -E -f) but it 
worked only locally. I noticed that the pacemaker process didn't started 
up normally on the node that was booting after the other. I also tried 
to remove files from /var/lib/pengine/ and /var/lib/hearbeat/crm/ but 
only the resources are gone. It didn't help on forming a cluster without 
resources. The pacemaker process exited some 20 minutes after it 
started. Manual starting was the same.

After digging into google for answers I found nothing helpful. From 
running tips I changed in the /etc/corosync/service.d/pcmk file the 
version to 1.1 (this is the version of the pacemaker in this distro). I 
realized that the cluster processes were startup from corosync itself 
not by pacemaker. Which could be omitted. The cluster forming is stable 
after this change even after many many events.

Now I reread the document mentioned above, and I wonder why it wrote the 
"Important notice" on page 37. What is wrong theoretically with my 
scenario? Why does it working? Why didn't work the config suggested by 
the document?

Tests were done firsth on virtual machines of a Fedora 14 (1 CPU core, 
512Mb ram, 10G disk, 1G drbd on logical volume, physical  volume on drbd 
forming volgroup named cluster.)/node.

Then on real machines. They have more cpu cores (4), more RAM (4G) and 
more disk (mirrored 750G), 180G drbd, and 100M garanteed routed link 
between the nodes 5 hops away.

By the way how should one configure the corosync to work on multicast 
routed network? I had to create an openvpn tap link between the real 
nodes for working. The original config with public IP-s didn't worked. 
Is corosync equipped to cope with the multicast pim messages? Or it was 
a firewall issue.

Thanks in advance,
Bence