[Pacemaker] problem with colocating and ordering sets of resources

[Ronney Meier - Rorotec Informatik GmbH]

> > Hi all
> >
> > I have a 2 node setup with lvm on top oft wo drbd devices and two xen
> virtual machines running.
> > The pacemaker version used is 1.0.11 .
> > All of the resources should always be running on the same node.
> > This usually works, but last week after a crash it happened, that pacemaker
> made one of the drbd devices master on one node, and the other drbd
> device on the other node. So it couldn't bring up the lvm anymore and
> therefore also none of the other resources were able to run.
> > My colocation and order constraints look like that here:
> >
> > colocation xen_on_drbd inf: ( xen-windows_server_sb
> > xen-windows_server_standard ) fs-xendata lv-drbd_data (
> > ms-drbd_r0:Master ms-drbd_r1:Master )
> >
> > order ord-xen_after_drbd inf: ( ms-drbd_r0:promote ms-drbd_r1:promote
> > ) lv-drbd_data:start fs-xendata:start ( xen-windows_server_sb:start
> > xen-windows_server_standard:start )
> >
> > So that it should first promote the two drbd devices, then den bringing up
> the lvm (=lv-drbd_data), then mounting the filesystem with the xen images
> (=fs_xendata) and then starting up the virtual machines.
> > Resources in the colocation constraint inside of parenthesis should be
> independent from each other.
> >
> > If I didn't completely missunderstood how the colocation of sets works, it
> should be impossible, that pacemaker promotes the two drbd devices on
> different nodes.
> > Or did I make an error with these constraints?
> 
> Yes, you make an error ;-) ... if you set the parenthesis around resources you
> set the sequential attribute to false. For an order resource sets that means:
> unordered. For a colocation set it means: not colocated ... and that is what
> you saw in you setup.
> 
> 
> Remove the parenthesis around the DRBD masters in your colocation and
> everything should be fine.
> 
> Are these two DRBD devices acting as two PVs for one VG? This is not a
> recommended setup because ... as you already saw ... the two DRBD master
> could end on different nodes or be disconnected individually having a
> different data generation in case of an failover and then you have serious
> problems with your VG. This can be handled properly in DRBD 8.4.x. that
> supports serveral volumes per connection.
> 
> Regards,
> Andreas

Hi andreas

Thanks a lot for your answer.

I was aware about what the sequential attribute means for an order resource...
>From the documentation I understood that for the colocation attribute it means that they are collocated, but don't depend on each other, so if one of them doesn't run pacemaker will not also shoot down the other ones (especially important for the virtual machines).
Ok, so I will take away the parenthesis, but just for further understanding:
If the parenthesis are around the drbd resources, it means they don't need to be collocated with each other. But don't they still need to be colocated with the lv-drbd_data resource? And since both of them need to be collocated with this resource they implicitly also need to be collocated with each other? Or pacemaker doesn't resolve this kind of implicit dependencies? (or another misunderstanding from my site?)

Yes, these two drbd devices act as two PV for one VG. We didn't had enough HDD space left, so we had to add more...
At least the problem with ending up on different nodes pacemaker should take care of (if properly configured, ähm), but if they get disconnected individually we'd had quite a big mess and I'm not even sure if the lvm driver would notice that. Thanks a lot, I didn't think about that. I Will see how I get drbd 8.4 running in debian...

ronney