<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=windows-1251"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi.<br>
<br>
You have hit this:<br>
<br>
<pre wrap="">Mar 3 16:49:16 breadnut2 VirtualDomain[20709]: INFO: Virtual domain vg.test1 currently has no state, retrying.
Mar 3 16:49:16 breadnut2 lrmd: [20694]: WARN: p-vd_vg.test1:monitor process (PID 20709) timed out (try 1). Killing with signal SIGTERM (15).
Mar 3 16:49:16 breadnut2 lrmd: [20694]: WARN: operation monitor[5] on ocf::VirtualDomain::p-vd_vg.test1 for client 20697, its parameters: crm_feature_set=[3.0.5] config=[/etc/libvirt/qemu/vg.test1.xml] CRM_meta_timeout=[20000] migration_transport=[tcp] : pid [20709] timed out
Mar 3 16:49:16 breadnut2 crmd: [20697]: ERROR: process_lrm_event: LRM operation p-vd_vg.test1_monitor_0 (5) Timed Out (timeout=20000ms)
</pre>
<br>
When a cluster node comes up, it is directed to probe each clustered
resource on the node. This behaviour does not depend on constraints,
this check is mandatory.<br>
<br>
At the moment, libvirtd is not running yet. Thus, VirtualDomain RA
is unable to connect to it and to check if your VM is running. So it
times out after some time.<br>
<br>
Timeout of monitor action implies "unknown error" of the resource.
Pengine cannot ensure that your resource is not running, so it
believes it is, and stops the resource everywhere, then starts it
again to recover.<br>
<br>
This is what you get. How to work around is a different story.
Frankly, I don't see a decent way.<br>
<br>
VirtualDomain RA really cannot tell if VM is running while it cannot
connect to libvirtd. I'm not too sure, but your log suggests that
libvirtd will not be started until VirtualDomain monitor returns.<br>
<br>
I'd suggest you to start libvirtd before corosync, from initscripts,
and see if it helps.<br>
<br>
May anyone propose a cleaner solution?<br>
<br>
<br>
--<br>
Pavel Levshin<br>
<br>
<br>
03.03.2011 9:05, AP пишет:
<blockquote cite="mid:20110303060539.GA2172@inml.weebeastie.net"
type="cite">
<pre wrap="">Hi,
Having deep issues with my cluster setup. Everything works ok until
I add a VirtualDomain RA in. Then things go pearshaped in that it seems
to ignore the "order" crm config for it and starts as soon as it can.
The crm config is provided below. Basically p-vd_vg.test1 attempts to
start despite p-libvirtd not being started and p-drbd_vg.test1 not
being master (or slave for that matter - ie it's not configured at all).
Eventually p-libvirtd and p-drbd_vg.test1 start and p-vd_vg.test1 attempts
to, pengine on the node where p-vd_vg.test1 is already running complains
with:
Mar 3 16:49:16 breadnut pengine: [2097]: ERROR: native_create_actions: Resource p-vd_vg.test1 (ocf::VirtualDomain) is active on 2 nodes attempting recovery
Mar 3 16:49:16 breadnut pengine: [2097]: WARN: See <a class="moz-txt-link-freetext" href="http://clusterlabs.org/wiki/FAQ#Resource_is_Too_Active">http://clusterlabs.org/wiki/FAQ#Resource_is_Too_Active</a> for more information.
Then mass slaughter occurs and p-vd_vg.test1 is restarted where it was
running previously whilst the other node gets an error for it.
Essentially I cannot restart the 2nd node without it breaking the 1st.
Now, as I understand it, a lone primitive will run once on any node - this
is just fine by me.
colo-vd_vg.test1 indicates that p-vd_vg.test1 should run where ms-drbd_vg.test1
is master. ms-drbd_vg.test1 should only be master where clone-libvirtd is
started.
order-vg.test1 indicates that ms-drbd_vg.test1 should start after clone-lvm_gh
is started (successfully). (This used to have a promote for ms-drbd_vg.test1
but then ms-drbd_vg.test1 would be demoted and not stopped on shutdown which
would cause clone-lvm_gh to error out on stop)
order-vd_vg.test1 indicates p-vd_vg.test1 should only start where
ms-drbd_vg.test1 and clone-libvirtd have both successfully started (the
order of their starting being irrelevant).
cli-standby-p-vd_vg.test1 was put there by my migrating p-vd_vg.test1
about the place.
This happens with or without fencing and with fencing configured as below
or as just a single primited with both nodes in the hostlist.
Help with this would be awesome and appreciated. I do not know what I am
missing here. The config makes sense to me so I don't even know where
to start poking and prodding. I be flailing.
Config and s/w version list is below:
OS: Debian Squeeze
Kernel: 2.6.37.2
PACKAGES:
ii cluster-agents 1:1.0.4-0ubuntu1~custom1 The reusable cluster components for Linux HA
ii cluster-glue 1.0.7-3ubuntu1~custom1 The reusable cluster components for Linux HA
ii corosync 1.3.0-1ubuntu1~custom1 Standards-based cluster framework (daemon and modules)
ii libccs3 3.1.0-0ubuntu1~custom1 Red Hat cluster suite - cluster configuration libraries
ii libcib1 1.1.5-0ubuntu1~ppa1~custom1 The Pacemaker libraries - CIB
ii libcman3 3.1.0-0ubuntu1~custom1 Red Hat cluster suite - cluster manager libraries
ii libcorosync4 1.3.0-1ubuntu1~custom1 Standards-based cluster framework (libraries)
ii libcrmcluster1 1.1.5-0ubuntu1~ppa1~custom1 The Pacemaker libraries - CRM
ii libcrmcommon2 1.1.5-0ubuntu1~ppa1~custom1 The Pacemaker libraries - common CRM
ii libfence4 3.1.0-0ubuntu1~custom1 Red Hat cluster suite - fence client library
ii liblrm2 1.0.7-3ubuntu1~custom1 Reusable cluster libraries -- liblrm2
ii libpe-rules2 1.1.5-0ubuntu1~ppa1~custom1 The Pacemaker libraries - rules for P-Engine
ii libpe-status3 1.1.5-0ubuntu1~ppa1~custom1 The Pacemaker libraries - status for P-Engine
ii libpengine3 1.1.5-0ubuntu1~ppa1~custom1 The Pacemaker libraries - P-Engine
ii libpils2 1.0.7-3ubuntu1~custom1 Reusable cluster libraries -- libpils2
ii libplumb2 1.0.7-3ubuntu1~custom1 Reusable cluster libraries -- libplumb2
ii libplumbgpl2 1.0.7-3ubuntu1~custom1 Reusable cluster libraries -- libplumbgpl2
ii libstonith1 1.0.7-3ubuntu1~custom1 Reusable cluster libraries -- libstonith1
ii libstonithd1 1.1.5-0ubuntu1~ppa1~custom1 The Pacemaker libraries - stonith
ii libtransitioner1 1.1.5-0ubuntu1~ppa1~custom1 The Pacemaker libraries - transitioner
ii pacemaker 1.1.5-0ubuntu1~ppa1~custom1 HA cluster resource manager
CONFIG:
node breadnut
node breadnut2 \
attributes standby="off"
primitive fencing-bn stonith:meatware \
params hostlist="breadnut" \
op start interval="0" timeout="60s" \
op stop interval="0" timeout="70s" \
op monitor interval="10" timeout="60s"
primitive fencing-bn2 stonith:meatware \
params hostlist="breadnut2" \
op start interval="0" timeout="60s" \
op stop interval="0" timeout="70s" \
op monitor interval="10" timeout="60s"
primitive p-drbd_vg.test1 ocf:linbit:drbd \
params drbd_resource="vg.test1" \
operations $id="ops-drbd_vg.test1" \
op start interval="0" timeout="240s" \
op stop interval="0" timeout="100s" \
op monitor interval="20" role="Master" timeout="20s" \
op monitor interval="30" role="Slave" timeout="20s"
primitive p-libvirtd ocf:local:libvirtd \
meta allow-migrate="off" \
op start interval="0" timeout="200s" \
op stop interval="0" timeout="100s" \
op monitor interval="10" timeout="200s"
primitive p-lvm_gh ocf:heartbeat:LVM \
params volgrpname="gh" \
meta allow-migrate="off" \
op start interval="0" timeout="90s" \
op stop interval="0" timeout="100s" \
op monitor interval="10" timeout="100s"
primitive p-vd_vg.test1 ocf:heartbeat:VirtualDomain \
params config="/etc/libvirt/qemu/vg.test1.xml" \
params migration_transport="tcp" \
meta allow-migrate="true" is-managed="true" \
op start interval="0" timeout="120s" \
op stop interval="0" timeout="120s" \
op migrate_to interval="0" timeout="120s" \
op migrate_from interval="0" timeout="120s" \
op monitor interval="10s" timeout="120s"
ms ms-drbd_vg.test1 p-drbd_vg.test1 \
meta resource-stickines="100" notify="true" master-max="2" target-role="Master"
clone clone-libvirtd p-libvirtd \
meta interleave="true"
clone clone-lvm_gh p-lvm_gh \
meta interleave="true"
location cli-standby-p-vd_vg.test1 p-vd_vg.test1 \
rule $id="cli-standby-rule-p-vd_vg.test1" -inf: #uname eq breadnut2
location loc-fencing-bn fencing-bn -inf: breadnut
location loc-fencing-bn2 fencing-bn2 -inf: breadnut2
colocation colo-vd_vg.test1 inf: p-vd_vg.test1:Started ms-drbd_vg.test1:Master clone-libvirtd:Started
order order-vd_vg.test1 inf: ( ms-drbd_vg.test1:start clone-libvirtd:start ) p-vd_vg.test1:start
order order-vg.test1 inf: clone-lvm_gh:start ms-drbd_vg.test1:start
property $id="cib-bootstrap-options" \
dc-version="1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
cluster-infrastructure="openais" \
default-resource-stickiness="1000" \
stonith-enabled="true" \
expected-quorum-votes="2" \
no-quorum-policy="ignore" \
last-lrm-refresh="1299128317"
</pre>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
Pacemaker mailing list: <a class="moz-txt-link-abbreviated" href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a>
<a class="moz-txt-link-freetext" href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a>
Project Home: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org">http://www.clusterlabs.org</a>
Getting started: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a>
Bugs: <a class="moz-txt-link-freetext" href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a>
</pre>
</blockquote>
</body>
</html>