[Pacemaker] LVM restarts after SLES upgrade
emmanuel segura
emi2fast at gmail.com
Tue Apr 24 06:08:05 EDT 2012
Hello Frank
If you would like to see what happening when the lvm hungs, put your
lvm configuration in DEBUG MODE
man lvm.conf
2012/4/24, Frank Meier <frank.meier at hr-group.de>:
> Oh, you are right, but this is an mistake by reconfigure the
> testszenario. At the original installation it was in the right order.
> Thanks
>
> Mit freundlichen Grüßen
>
> Frank Meier
> UNIX-Basis
>
> Hamm Reno Group GmbH
> Industriegebiet West | D-66987 Thaleischweiler-Fröschen
> T.+49(0)6334 444-8322 | F.+49(0)6334 444-8190
> frank.meier at hr-group.de | www.reno.de
> ___________________________________________________________________
>
> Sitz: Am Tie 7 | D-49086 Osnabrück
> Handelsregister Osnabrück HRB 19587
> Geschäftsführer: Hans-Jürgen de Fries,
> Jens Gransee, Manfred Klumpp,
> Robert Reisch
>
> Am 24.04.2012 11:28, schrieb pacemaker-request at oss.clusterlabs.org:
>> Send Pacemaker mailing list submissions to
>> pacemaker at oss.clusterlabs.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>> or, via email, send a message with subject or body 'help' to
>> pacemaker-request at oss.clusterlabs.org
>>
>> You can reach the person managing the list at
>> pacemaker-owner at oss.clusterlabs.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of Pacemaker digest..."
>>
>>
>> Today's Topics:
>>
>> 1. Re: LVM restarts after SLES upgrade (emmanuel segura)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Tue, 24 Apr 2012 09:34:12 +0000
>> From: emmanuel segura <emi2fast at gmail.com>
>> To: The Pacemaker cluster resource manager
>> <pacemaker at oss.clusterlabs.org>
>> Subject: Re: [Pacemaker] LVM restarts after SLES upgrade
>> Message-ID:
>>
>> <CAE7pJ3DGvkzMR1d9HNm76s8wtrAj0t1==31RnafHe-YEgTV06Q at mail.gmail.com>
>> Content-Type: text/plain; charset=ISO-8859-1
>>
>> Hello Frank
>>
>> Maybe this it's not the probelem, but i see this constrain wrong from
>> my point of view
>> =============================================================
>> order o-Testclustervm inf: c-xen-vg-fs vm-clusterTest
>> order o-clvmglue-xenvgfs inf: c-clvm-glue c-xen-vg-fs
>> =============================================================
>> to be
>> =============================================================
>> order o-clvmglue-xenvgfs inf: c-clvm-glue c-xen-vg-fs
>> order o-Testclustervm inf: c-xen-vg-fs vm-clusterTest
>> =============================================================
>> 2012/4/24, Frank Meier <frank.meier at hr-group.de>:
>>> Hi,
>>>
>>> Every time the vgdisplay -v TestXenVG is hanging(ca.2min)
>>>
>>> I see two of this peocesses:
>>> /bin/sh /usr/lib/ocf/resource.d//heartbeat/LVM monitor
>>> /bin/sh /usr/lib/ocf/resource.d//heartbeat/LVM monitor
>>> is this OK, or have we a race condition?
>>>
>>>
>>>
>>> Hi,
>>>
>>> I do a new try.
>>>
>>> I've moved our productive VMs to another server and reconfigure the
>>> cluster. But the problem is the same: every hour(or shorter, it seemd to
>>> be full, half or quarter hour), the LVM is restarted.
>>> Last I saw vgdisplay -v TestXenVG hanging, I don't if this was every
>>> time.
>>>
>>> I hope anybody can help. It has to be run expressly.
>>>
>>> Thanks
>>>
>>> Frank
>>>
>>>
>>> vgs
>>> VG #PV #LV #SN Attr VSize VFree
>>> TestXenVG 2 2 0 wz--nc 67.99g 61.80g
>>>
>>> xencluster1:~ # ps aux|grep clvm
>>> root 21529 0.0 0.0 4520 528 pts/2 S+ 17:36 0:00 grep
>>> clvm
>>> root 21756 0.0 0.1 155348 62920 ? SLsl 13:44 0:07
>>> /usr/sbin/clvmd -d0
>>>
>>>
>>> xencluster1:~ # time vgdisplay -v TestXenVG
>>> Using volume group(s) on command line
>>> Finding volume group "TestXenVG"
>>> --- Volume group ---
>>> VG Name TestXenVG
>>> System ID
>>> Format lvm2
>>> Metadata Areas 2
>>> Metadata Sequence No 6
>>> VG Access read/write
>>> VG Status resizable
>>> Clustered yes
>>> Shared no
>>> MAX LV 0
>>> Cur LV 2
>>> Open LV 2
>>> Max PV 0
>>> Cur PV 2
>>> Act PV 2
>>> VG Size 67.99 GiB
>>> PE Size 4.00 MiB
>>> Total PE 17406
>>> Alloc PE / Size 1586 / 6.20 GiB
>>> Free PE / Size 15820 / 61.80 GiB
>>> VG UUID hveP8R-8pXh-b9QB-OGKG-ZmPa-xXXQ-f3MYGx
>>>
>>> --- Logical volume ---
>>> LV Name /dev/TestXenVG/vm1
>>> VG Name TestXenVG
>>> LV UUID W01y7D-7WfV-bBiU-pmnm-Tecg-ecrb-qjJ7GJ
>>> LV Write Access read/write
>>> LV Status available
>>> # open 1
>>> LV Size 6.00 GiB
>>> Current LE 1536
>>> Segments 1
>>> Allocation inherit
>>> Read ahead sectors auto
>>> - currently set to 1024
>>> Block device 253:4
>>>
>>> --- Logical volume ---
>>> LV Name /dev/TestXenVG/xen_config
>>> VG Name TestXenVG
>>> LV UUID lrEjLC-yrlv-Kddh-JIb8-A1pT-t6b1-W4dw3f
>>> LV Write Access read/write
>>> LV Status available
>>> # open 1
>>> LV Size 200.00 MiB
>>> Current LE 50
>>> Segments 1
>>> Allocation inherit
>>> Read ahead sectors auto
>>> - currently set to 1024
>>> Block device 253:5
>>>
>>> --- Physical volumes ---
>>> PV Name /dev/mapper/mpathi
>>> PV UUID N7NE4R-v1ji-23cS-NzIn-v6Zt-S3Rp-HlB5jS
>>> PV Status allocatable
>>> Total PE / Free PE 8703 / 7117
>>>
>>> PV Name /dev/mapper/mpathj
>>> PV UUID cR567j-PM7y-5NUN-SGsJ-JO2h-rn0e-jTk7sm
>>> PV Status allocatable
>>> Total PE / Free PE 8703 / 8703
>>>
>>>
>>> real 0m0.033s
>>> user 0m0.004s
>>> sys 0m0.016s
>>>
>>> CIB:
>>>
>>> node xencluster1 \
>>> attributes standby="off"
>>> node xencluster2 \
>>> attributes standby="off"
>>> primitive STONITH-Node1 stonith:meatware \
>>> meta target-role="Started" is-managed="true" \
>>> operations $id="STONITH-Node1-operations" \
>>> op monitor interval="3600" timeout="20" start-delay="15" \
>>> params hostlist="xencluster1"
>>> primitive STONITH-Node2 stonith:meatware \
>>> meta target-role="Started" is-managed="true" \
>>> operations $id="STONITH-Node2-operations" \
>>> op monitor interval="3600" timeout="20" start-delay="15" \
>>> params hostlist="xencluster2"
>>> primitive clvm ocf:lvm2:clvmd \
>>> operations $id="clvm-operations" \
>>> op monitor interval="10" timeout="20" \
>>> op start interval="0" timeout="90" \
>>> op stop interval="0" timeout="100"
>>> primitive clvm-xenvg ocf:heartbeat:LVM \
>>> operations $id="clvm-xenvg-operations" \
>>> op monitor interval="10" timeout="90" \
>>> op start interval="0" timeout="60" \
>>> op stop interval="0" timeout="60" \
>>> params volgrpname="TestXenVG" \
>>> meta target-role="Started"
>>> primitive cmirror ocf:lvm2:cmirrord \
>>> operations $id="cmirror-operations" \
>>> op monitor interval="10" timeout="20" \
>>> op start interval="0" timeout="90" \
>>> op stop interval="0" timeout="100" \
>>> params daemon_timeout="30"
>>> primitive dlm ocf:pacemaker:controld \
>>> operations $id="dlm-operations" \
>>> op monitor interval="60" timeout="60" start-delay="0" \
>>> op start interval="0" timeout="90" \
>>> op stop interval="0" timeout="100"
>>> primitive fs-config-xen ocf:heartbeat:Filesystem \
>>> operations $id="fs-config-xen-operations" \
>>> op monitor interval="20" timeout="40" \
>>> params device="/dev/mapper/TestXenVG-xen_config"
>>> directory="/etc/xen/vm" fstype="ocfs2"
>>> primitive o2cb ocf:ocfs2:o2cb \
>>> operations $id="o2cb-operations" \
>>> op monitor interval="60" timeout="60" \
>>> op start interval="0" timeout="90" \
>>> op stop interval="0" timeout="100"
>>> primitive vm-clusterTest ocf:heartbeat:Xen \
>>> meta target-role="Started" allow-migrate="true" is-managed="true"
>>> \
>>> operations $id="vm-clusterTest-operations" \
>>> op monitor interval="10" timeout="30" \
>>> op start interval="0" timeout="60" \
>>> op stop interval="0" timeout="60" \
>>> op migrate_to interval="0" timeout="360" \
>>> params xmfile="/etc/xen/vm/ClusterTest"
>>> group clvm-glue dlm clvm o2cb cmirror
>>> group xen-vg-fs clvm-xenvg fs-config-xen
>>> clone c-clvm-glue clvm-glue \
>>> meta target-role="Started" interleave="true" ordered="true"
>>> clone c-xen-vg-fs xen-vg-fs \
>>> meta target-role="Started" interleave="true" ordered="true"
>>> is-managed="true"
>>> location STONITHnode1Constraint STONITH-Node1 -inf: xencluster1
>>> location STONITHnode2Constraint STONITH-Node2 -inf: xencluster2
>>> colocation colo-clvmglue-xenvgsfs inf: c-xen-vg-fs c-clvm-glue
>>> order o-Testclustervm inf: c-xen-vg-fs vm-clusterTest
>>> order o-clvmglue-xenvgfs inf: c-clvm-glue c-xen-vg-fs
>>> property $id="cib-bootstrap-options" \
>>> dc-version="1.1.6-b988976485d15cb702c9307df55512d323831a5e" \
>>> no-quorum-policy="ignore" \
>>> default-resource-stickiness="100000" \
>>> last-lrm-refresh="1335186478" \
>>> cluster-infrastructure="openais" \
>>> expected-quorum-votes="2" \
>>> default-action-timeout="120s" \
>>> maintenance-mode="false"
>>> op_defaults $id="op_defaults-options" \
>>> record-pending="false"
>>>
>>>
>>>
>>> Log:
>>> Apr 23 17:14:29 xencluster1 clvmd[21756]: Syncing device names
>>> Apr 23 17:14:29 xencluster1 clvmd[21756]: Reply from node 4914420a: 0
>>> bytes
>>> Apr 23 17:14:29 xencluster1 clvmd[21756]: Got 1 replies, expecting: 2
>>> Apr 23 17:14:29 xencluster1 clvmd[21756]: LVM thread waiting for work
>>> Apr 23 17:15:59 xencluster1 lrmd: [8619]: WARN: clvm-xenvg:0:monitor
>>> process (PID 8351) timed out (try 1). Killing with signal SIGTERM (15).
>>> Apr 23 17:15:59 xencluster1 lrmd: [8619]: WARN: operation monitor[339]
>>> on clvm-xenvg:0 for client 8622: pid 8351 timed out
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: ERROR: process_lrm_event: LRM
>>> operation clvm-xenvg:0_monitor_10000 (339) Timed Out (timeout=90000ms)
>>> pr 23 17:15:59 xencluster1 crmd: [8622]: info: process_graph_event:
>>> Action clvm-xenvg:0_monitor_10000 arrived after a completed transition
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: info: abort_transition_graph:
>>> process_graph_event:481 - Triggered transition abort (complete=1,
>>> tag=lrm_rsc_op, id=clvm-xenvg:0_monitor_10000,
>>> magic=2:-2;12:384:0:a2db503d-6be0-4ed9-a94c-0e7031c748ad, cib=0.8361.49)
>>> : Inactive graph
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: WARN: update_failcount:
>>> Updating failcount for clvm-xenvg:0 on xencluster1 after failed monitor:
>>> rc=-2 (update=value++, time=1335194159)
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: info: do_state_transition:
>>> State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC
>>> cause=C_FSA_INTERNAL origin=abort_transition_graph ]
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: info: do_state_transition: All
>>> 2 cluster nodes are eligible to run resources.
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: info: do_pe_invoke: Query
>>> 2608: Requesting the current CIB: S_POLICY_ENGINE
>>> Apr 23 17:15:59 xencluster1 attrd: [8620]: notice: attrd_trigger_update:
>>> Sending flush op to all hosts for: fail-count-clvm-xenvg:0 (8)
>>> Apr 23 17:15:59 xencluster1 attrd: [8620]: notice: attrd_perform_update:
>>> Sent update 1231: fail-count-clvm-xenvg:0=8
>>> Apr 23 17:15:59 xencluster1 attrd: [8620]: notice: attrd_trigger_update:
>>> Sending flush op to all hosts for: last-failure-clvm-xenvg:0 (1335194159)
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: info: do_pe_invoke_callback:
>>> Invoking the PE: query=2608, ref=pe_calc-dc-1335194159-1765, seq=2556,
>>> quorate=1
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: info: abort_transition_graph:
>>> te_update_diff:176 - Triggered transition abort (complete=1, tag=nvpair,
>>> id=status-xencluster1-fail-count-clvm-xenvg.0,
>>> name=fail-count-clvm-xenvg:0, value=8, magic=NA, cib=0.8361.50) :
>>> Transient attribute: update
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: info: do_pe_invoke: Query
>>> 2609: Requesting the current CIB: S_POLICY_ENGINE
>>> Apr 23 17:15:59 xencluster1 attrd: [8620]: notice: attrd_perform_update:
>>> Sent update 1233: last-failure-clvm-xenvg:0=1335194159
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: unpack_config: On
>>> loss of CCM Quorum: Ignore
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: unpack_rsc_op:
>>> Operation STONITH-Node1_last_failure_0 found resource STONITH-Node1
>>> active on xencluster2
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: unpack_rsc_op:
>>> Operation dlm:1_last_failure_0 found resource dlm:1 active on xencluster2
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: unpack_rsc_op:
>>> Operation clvm:1_last_failure_0 found resource clvm:1 active on
>>> xencluster2
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: unpack_rsc_op:
>>> Operation cmirror:1_last_failure_0 found resource cmirror:1 active on
>>> xencluster2
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: unpack_rsc_op:
>>> Operation clvm:0_last_failure_0 found resource clvm:0 active on
>>> xencluster1
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: unpack_rsc_op:
>>> Operation STONITH-Node2_last_failure_0 found resource STONITH-Node2
>>> active on xencluster1
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: unpack_rsc_op:
>>> Operation dlm:0_last_failure_0 found resource dlm:0 active on xencluster1
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: unpack_rsc_op:
>>> Operation clvm-xenvg:0_last_failure_0 found resource clvm-xenvg:0 active
>>> on xencluster1
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: WARN: unpack_rsc_op:
>>> Processing failed op clvm-xenvg:0_monitor_10000 on xencluster1: unknown
>>> exec error (-2)
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: unpack_rsc_op:
>>> Operation cmirror:0_last_failure_0 found resource cmirror:0 active on
>>> xencluster1
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: unpack_rsc_op:
>>> Operation o2cb:0_last_failure_0 found resource o2cb:0 active on
>>> xencluster1
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice:
>>> common_apply_stickiness: c-xen-vg-fs can fail 999993 more times on
>>> xencluster1 before being forced off
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice:
>>> common_apply_stickiness: c-xen-vg-fs can fail 999993 more times on
>>> xencluster1 before being forced off
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice:
>>> common_apply_stickiness: c-xen-vg-fs can fail 999993 more times on
>>> xencluster2 before being forced off
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice:
>>> common_apply_stickiness: c-xen-vg-fs can fail 999993 more times on
>>> xencluster2 before being forced off
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: info: do_pe_invoke_callback:
>>> Invoking the PE: query=2609, ref=pe_calc-dc-1335194159-1766, seq=2556,
>>> quorate=1
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: RecurringOp: Start
>>> recurring monitor (10s) for clvm-xenvg:0 on xencluster1
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: info: abort_transition_graph:
>>> te_update_diff:176 - Triggered transition abort (complete=1, tag=nvpair,
>>> id=status-xencluster1-last-failure-clvm-xenvg.0,
>>> name=last-failure-clvm-xenvg:0, value=1335194159, magic=NA,
>>> cib=0.8361.51) : Transient attribute: update
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: LogActions: Recover
>>> clvm-xenvg:0 (Started xencluster1)
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: LogActions: Restart
>>> fs-config-xen:0 (Started xencluster1)
>>> Apr 23 17:15:59 xencluster1 crmd: [8622]: info: do_pe_invoke: Query
>>> 2610: Requesting the current CIB: S_POLICY_ENGINE
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: LogActions: Leave
>>> clvm-xenvg:1 (Started xencluster2)
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: LogActions: Leave
>>> fs-config-xen:1 (Started xencluster2)
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: LogActions: Leave
>>> STONITH-Node1 (Started xencluster2)
>>> Apr 23 17:15:59 xencluster1 pengine: [8621]: notice: LogActions: Leave
>>> STONITH-Node2 (Started xencluster1)
>>>
>>>
>>>
>>>
>>>
>>>
>>> Mit freundlichen Gr??en
>>>
>>> Frank Meier
>>> UNIX-Basis
>>>
>>> Hamm Reno Group GmbH
>>> Industriegebiet West | D-66987 Thaleischweiler-Fr?schen
>>> T.+49(0)6334 444-322 | F.+49(0)6334 444-190
>>> frank.meier at hr-group.de | www.my-reno.com
>>> ___________________________________________________________________
>>>
>>> Sitz: Am Tie 7 | D-49086 Osnabr?ck
>>> Handelsregister Osnabr?ck HRB 19587
>>> Gesch?ftsf?hrer: Hans-J?rgen de Fries,
>>> Jens Gransee, Manfred Klumpp,
>>> Robert Reisch, Albert Scherer
>>> ________________________________________
>>> Von: pacemaker-request at oss.clusterlabs.org
>>> [pacemaker-request at oss.clusterlabs.org]
>>> Gesendet: Donnerstag, 19. April 2012 10:07
>>> An: pacemaker at oss.clusterlabs.org
>>> Betreff: Pacemaker Digest, Vol 53, Issue 42
>>>
>>> Send Pacemaker mailing list submissions to
>>> pacemaker at oss.clusterlabs.org
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>> or, via email, send a message with subject or body 'help' to
>>> pacemaker-request at oss.clusterlabs.org
>>>
>>> You can reach the person managing the list at
>>> pacemaker-owner at oss.clusterlabs.org
>>>
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of Pacemaker digest..."
>>>
>>>
>>> Today's Topics:
>>>
>>> 1. Re: Pacemaker Digest, Vol 53, Issue 40 (emmanuel segura)
>>>
>>>
>>> ----------------------------------------------------------------------
>>>
>>> Message: 1
>>> Date: Thu, 19 Apr 2012 10:12:37 +0200
>>> From: emmanuel segura <emi2fast at gmail.com>
>>> To: The Pacemaker cluster resource manager
>>> <pacemaker at oss.clusterlabs.org>
>>> Subject: Re: [Pacemaker] Pacemaker Digest, Vol 53, Issue 40
>>> Message-ID:
>>>
>>> <CAE7pJ3B5L1m2s=OQ=cUDd9=mqy50ORhP+eiDteCRp5_6JLR6jQ at mail.gmail.com>
>>> Content-Type: text/plain; charset="iso-8859-1"
>>>
>>> vgchange -cy XenVG
>>>
>>> your vg it's not clustered and check the clvmd process
>>>
>>> ps -ef | grep clvmd
>>>
>>> Il giorno 19 aprile 2012 09:45, Frank Meier <frank.meier at hr-group.de> ha
>>> scritto:
>>>
>>>> xencluster1:~ # vgs
>>>> VG #PV #LV #SN Attr VSize VFree
>>>> XenVG 8 32 0 wz--n- 938.16g 530.98g
>>>>
>>>>
>>>> Mit freundlichen Gr??en
>>>>
>>>> Frank Meier
>>>> UNIX-Basis
>>>>
>>>> Hamm Reno Group GmbH
>>>> Industriegebiet West | D-66987 Thaleischweiler-Fr?schen
>>>> T.+49(0)6334 444-8322 | F.+49(0)6334 444-8190
>>>> frank.meier at hr-group.de | www.reno.de
>>>> ___________________________________________________________________
>>>>
>>>> Sitz: Am Tie 7 | D-49086 Osnabr?ck
>>>> Handelsregister Osnabr?ck HRB 19587
>>>> Gesch?ftsf?hrer: Hans-J?rgen de Fries,
>>>> Jens Gransee, Manfred Klumpp,
>>>> Robert Reisch
>>>>
>>>> Am 19.04.2012 09:32, schrieb pacemaker-request at oss.clusterlabs.org:
>>>>> Send Pacemaker mailing list submissions to
>>>>> pacemaker at oss.clusterlabs.org
>>>>>
>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>> or, via email, send a message with subject or body 'help' to
>>>>> pacemaker-request at oss.clusterlabs.org
>>>>>
>>>>> You can reach the person managing the list at
>>>>> pacemaker-owner at oss.clusterlabs.org
>>>>>
>>>>> When replying, please edit your Subject line so it is more specific
>>>>> than "Re: Contents of Pacemaker digest..."
>>>>>
>>>>>
>>>>> Today's Topics:
>>>>>
>>>>> 1. Re: Pacemaker Digest, Vol 53, Issue 38 (emmanuel segura)
>>>>>
>>>>>
>>>>> ----------------------------------------------------------------------
>>>>>
>>>>> Message: 1
>>>>> Date: Thu, 19 Apr 2012 09:37:31 +0200
>>>>> From: emmanuel segura <emi2fast at gmail.com>
>>>>> To: The Pacemaker cluster resource manager
>>>>> <pacemaker at oss.clusterlabs.org>
>>>>> Subject: Re: [Pacemaker] Pacemaker Digest, Vol 53, Issue 38
>>>>> Message-ID:
>>>>> <CAE7pJ3CovBAsjf+vbbJGEKuqTTvFiWWpn0S=UvOTSoyQpeTa=
>>>> w at mail.gmail.com>
>>>>> Content-Type: text/plain; charset="iso-8859-1"
>>>>>
>>>>> give this command
>>>>>
>>>>> vgs
>>>>>
>>>>> Il giorno 19 aprile 2012 09:32, Frank Meier <frank.meier at hr-group.de>
>>>>> ha
>>>>> scritto:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> here my lvm.conf:
>>>>>>
>>>>>> devices {
>>>>>> dir = "/dev"
>>>>>> scan = [ "/dev" ]
>>>>>> filter = [ "a|/dev/mapper|", "r/.*/" ]
>>>>>> cache = "/etc/lvm/.cache"
>>>>>> write_cache_state = 1
>>>>>> types = [ "device-mapper", 16 ]
>>>>>> sysfs_scan = 1
>>>>>> md_component_detection = 1
>>>>>> }
>>>>>> log {
>>>>>> verbose = 0
>>>>>> syslog = 1
>>>>>> overwrite = 0
>>>>>> level = 0
>>>>>> indent = 1
>>>>>> command_names = 0
>>>>>> prefix = " "
>>>>>> }
>>>>>> backup {
>>>>>> backup = 1
>>>>>> backup_dir = "/etc/lvm/backup"
>>>>>> archive = 1
>>>>>> archive_dir = "/etc/lvm/archive"
>>>>>> retain_min = 10
>>>>>> retain_days = 30
>>>>>> }
>>>>>> shell {
>>>>>> history_size = 100
>>>>>> }
>>>>>> global {
>>>>>> umask = 077
>>>>>> test = 0
>>>>>> activation = 1
>>>>>> proc = "/proc"
>>>>>> locking_type = 3
>>>>>> locking_dir = "/var/lock/lvm"
>>>>>> }
>>>>>> activation {
>>>>>> missing_stripe_filler = "error"
>>>>>> mirror_region_size = 512
>>>>>> reserved_stack = 256
>>>>>> reserved_memory = 8192
>>>>>> process_priority = -18
>>>>>> }
>>>>>> dmeventd {
>>>>>> mirror_library = "libdevmapper-event-lvm2mirror.so.2.02"
>>>>>> snapshot_library = "libdevmapper-event-lvm2snapshot.so.2.02"
>>>>>> }
>>>>>>
>>>>>>
>>>>>> Mit freundlichen Gr??en
>>>>>>
>>>>>> Frank Meier
>>>>>> UNIX-Basis
>>>>>>
>>>>>> Hamm Reno Group GmbH
>>>>>> Industriegebiet West | D-66987 Thaleischweiler-Fr?schen
>>>>>> T.+49(0)6334 444-8322 | F.+49(0)6334 444-8190
>>>>>> frank.meier at hr-group.de | www.reno.de
>>>>>> ___________________________________________________________________
>>>>>>
>>>>>> Sitz: Am Tie 7 | D-49086 Osnabr?ck
>>>>>> Handelsregister Osnabr?ck HRB 19587
>>>>>> Gesch?ftsf?hrer: Hans-J?rgen de Fries,
>>>>>> Jens Gransee, Manfred Klumpp,
>>>>>> Robert Reisch
>>>>>>
>>>>>> Am 19.04.2012 09:13, schrieb pacemaker-request at oss.clusterlabs.org:
>>>>>>> Send Pacemaker mailing list submissions to
>>>>>>> pacemaker at oss.clusterlabs.org
>>>>>>>
>>>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>> or, via email, send a message with subject or body 'help' to
>>>>>>> pacemaker-request at oss.clusterlabs.org
>>>>>>>
>>>>>>> You can reach the person managing the list at
>>>>>>> pacemaker-owner at oss.clusterlabs.org
>>>>>>>
>>>>>>> When replying, please edit your Subject line so it is more specific
>>>>>>> than "Re: Contents of Pacemaker digest..."
>>>>>>>
>>>>>>>
>>>>>>> Today's Topics:
>>>>>>>
>>>>>>> 1. Re: LVM restarts after SLES upgrade (emmanuel segura)
>>>>>>>
>>>>>>>
>>>>>>> ----------------------------------------------------------------------
>>>>>>>
>>>>>>> Message: 1
>>>>>>> Date: Thu, 19 Apr 2012 09:18:37 +0200
>>>>>>> From: emmanuel segura <emi2fast at gmail.com>
>>>>>>> To: The Pacemaker cluster resource manager
>>>>>>> <pacemaker at oss.clusterlabs.org>
>>>>>>> Subject: Re: [Pacemaker] LVM restarts after SLES upgrade
>>>>>>> Message-ID:
>>>>>>> <CAE7pJ3Ab1ZzSpou0=vvrO=
>>>>>> mb3ur2+K1XhALLJEpj-sOOUEfwTA at mail.gmail.com>
>>>>>>> Content-Type: text/plain; charset="iso-8859-1"
>>>>>>>
>>>>>>> maybe you have problem with clvm
>>>>>>>
>>>>>>> can you show me your vm configuration?
>>>>>>>
>>>>>>> Il giorno 19 aprile 2012 08:29, Frank Meier <frank.meier at hr-group.de>
>>>> ha
>>>>>>> scritto:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I've installed a 2-Node Xen-Cluster with SLES 11 SP1.
>>>>>>>>
>>>>>>>> After an upgrade to SLES11 SP2 the cluster won't work as the old
>>>>>>>> one.
>>>>>>>>
>>>>>>>> After some time, all LVs reorganize and so the VM where restarted.
>>>>>>>>
>>>>>>>> Any clue?
>>>>>>>>
>>>>>>>> CIB:
>>>>>>>>
>>>>>>>> node xencluster1 \
>>>>>>>> attributes standby="off"
>>>>>>>> node xencluster2 \
>>>>>>>> attributes standby="off"
>>>>>>>> primitive STONITH--Node1 stonith:meatware \
>>>>>>>> meta is-managed="true" target-role="Started" \
>>>>>>>> operations $id="STONITH--Node1-operations" \
>>>>>>>> op monitor interval="15" timeout="15" start-delay="15" \
>>>>>>>> params hostlist="xencluster1"
>>>>>>>> primitive STONITH-Node2 stonith:meatware \
>>>>>>>> meta target-role="started" is-managed="true" \
>>>>>>>> operations $id="STONITH-Node2-operations" \
>>>>>>>> op monitor interval="15" timeout="15" start-delay="15" \
>>>>>>>> params hostlist="xencluster2"
>>>>>>>> primitive clvm ocf:lvm2:clvmd \
>>>>>>>> operations $id="clvm-operations" \
>>>>>>>> op monitor interval="10" timeout="20" \
>>>>>>>> op start interval="0" timeout="90" \
>>>>>>>> op stop interval="0" timeout="100" \
>>>>>>>> params daemon_timeout="30"
>>>>>>>> primitive clvm-xenvg ocf:heartbeat:LVM \
>>>>>>>> operations $id="clvm-xenvg-operations" \
>>>>>>>> op monitor interval="10" timeout="30" \
>>>>>>>> params volgrpname="XenVG"
>>>>>>>> primitive cmirror ocf:lvm2:cmirrord \
>>>>>>>> operations $id="cmirror-operations" \
>>>>>>>> op monitor interval="10" timeout="20" \
>>>>>>>> op start interval="0" timeout="90" \
>>>>>>>> op stop interval="0" timeout="100" \
>>>>>>>> params daemon_timeout="30"
>>>>>>>> primitive dlm ocf:pacemaker:controld \
>>>>>>>> operations $id="dlm-operations" \
>>>>>>>> op monitor interval="10" timeout="20" start-delay="0" \
>>>>>>>> op start interval="0" timeout="90" \
>>>>>>>> op stop interval="0" timeout="100"
>>>>>>>> primitive fs-config-xen ocf:heartbeat:Filesystem \
>>>>>>>> meta is-managed="true" target-role="Started" \
>>>>>>>> operations $id="fs-config-xen-operations" \
>>>>>>>> op monitor interval="20" timeout="40" \
>>>>>>>> params device="/dev/mapper/XenVG-xenconfig"
>>>>>> directory="/etc/xen/vm"
>>>>>>>> fstype="ocfs2"
>>>>>>>> primitive o2cb ocf:ocfs2:o2cb \
>>>>>>>> operations $id="o2cb-operations" \
>>>>>>>> op monitor interval="10" timeout="20" \
>>>>>>>> op start interval="0" timeout="90" \
>>>>>>>> op stop interval="0" timeout="100"
>>>>>>>> primitive vm-cim ocf:heartbeat:Xen \
>>>>>>>> meta target-role="Started" is-managed="true"
>>>>>> allow-migrate="true" \
>>>>>>>> operations $id="vm-cim-operations" \
>>>>>>>> op monitor interval="10" timeout="30" \
>>>>>>>> op start interval="0" timeout="60" \
>>>>>>>> op stop interval="0" timeout="60" \
>>>>>>>> op migrate_to interval="0" timeout="360" \
>>>>>>>> params xmfile="/etc/xen/vm/cim"
>>>>>>>> primitive vm-deprepo ocf:heartbeat:Xen \
>>>>>>>> meta target-role="started" is-managed="true"
>>>>>> allow-migrate="true" \
>>>>>>>> operations $id="vm-deprepo-operations" \
>>>>>>>> op monitor interval="10" timeout="30" \
>>>>>>>> op start interval="0" timeout="60" \
>>>>>>>> op stop interval="0" timeout="60" \
>>>>>>>> op migrate_to interval="0" timeout="360" \
>>>>>>>> params xmfile="/etc/xen/vm/debrepo"
>>>>>>>>
>>>>>>>>
>>>>>>>> group clvm-glue dlm clvm o2cb cmirror \
>>>>>>>> meta target-role="Started"
>>>>>>>> group xen-vg-fs clvm-xenvg fs-config-xen
>>>>>>>> clone c-clvm-glue clvm-glue \
>>>>>>>> meta interleave="true" ordered="true" target-role="Started"
>>>>>>>> clone c-xen-vg-fs xen-vg-fs \
>>>>>>>> meta interleave="true" ordered="true" is-managed="true"
>>>>>>>> target-role="Started"
>>>>>>>> location STONITHnode1Constraint STONITH--Node1 -inf: xencluster1
>>>>>>>> colocation colo-clvmglue-xenvgsfs inf: c-xen-vg-fs c-clvm-glue
>>>>>>>> order o-cim inf: c-xen-vg-fs vm-cim
>>>>>>>> order o-clvmglue-xenvgfs inf: c-clvm-glue c-xen-vg-fs
>>>>>>>> order o-deprepo inf: c-xen-vg-fs vm-deprepo
>>>>>>>> order o-ilmt inf: c-xen-vg-fs vm-ilmt
>>>>>>>> order o-intranet inf: c-xen-vg-fs vm-intranet
>>>>>>>> order o-nagois inf: c-xen-vg-fs vm-nagios
>>>>>>>> order o-oneiroi inf: c-xen-vg-fs vm-oneiroi
>>>>>>>> order o-otbridge inf: c-xen-vg-fs vm-otbridge
>>>>>>>> order o-soa inf: c-xen-vg-fs vm-soa
>>>>>>>> order o-topdesk-entw inf: c-xen-vg-fs vm-topdesk-entw
>>>>>>>> order o-traumschiff inf: c-xen-vg-fs vm-traumschiff
>>>>>>>> order o-virenscanner inf: c-xen-vg-fs vm-virenscanner
>>>>>>>> property $id="cib-bootstrap-options" \
>>>>>>>> dc-version="1.1.6-b988976485d15cb702c9307df55512d323831a5e" \
>>>>>>>> no-quorum-policy="ignore" \
>>>>>>>> default-resource-stickiness="100000" \
>>>>>>>> last-lrm-refresh="1334518148" \
>>>>>>>> cluster-infrastructure="openais" \
>>>>>>>> expected-quorum-votes="2" \
>>>>>>>> default-action-timeout="120s" \
>>>>>>>> maintenance-mode="true"
>>>>>>>> op_defaults $id="op_defaults-options" \
>>>>>>>> record-pending="false"
>>>>>>>>
>>>>>>>> Node 1Logs:
>>>>>>>>
>>>>>>>> Apr 15 22:01:10 xencluster1 clvmd[8763]: Sending message to all
>>>> cluster
>>>>>>>> nodes
>>>>>>>> Apr 15 22:01:10 xencluster1 clvmd[8763]: process_work_item: local
>>>>>>>> Apr 15 22:01:10 xencluster1 clvmd[8763]: process_local_command:
>>>> unknown
>>>>>>>> (0x2d) msg=0x7f5c40010e80, msglen =32, client=0x7f5c400578d0
>>>>>>>> Apr 15 22:01:10 xencluster1 clvmd[8763]: Syncing device names
>>>>>>>> Apr 15 22:01:10 xencluster1 clvmd[8763]: Reply from node 4914420a: 0
>>>>>> bytes
>>>>>>>> Apr 15 22:01:10 xencluster1 clvmd[8763]: Got 1 replies, expecting: 2
>>>>>>>> Apr 15 22:01:10 xencluster1 clvmd[8763]: LVM thread waiting for work
>>>>>>>> Apr 15 22:01:10 xencluster1 clvmd[8763]: 1226064394 got message from
>>>>>>>> nodeid 1226064394 for 0. len 32
>>>>>>>> Apr 15 22:01:40 xencluster1 lrmd: [8093]: WARN: clvm-xenvg:0:monitor
>>>>>>>> process (PID 31785) timed out (try 1). Killing with signal SIGTERM
>>>>>> (15).
>>>>>>>> Apr 15 22:01:40 xencluster1 lrmd: [8093]: WARN: operation
>>>>>>>> monitor[92]
>>>> on
>>>>>>>> clvm-xenvg:0 for client 8096: pid 31785 timed out
>>>>>>>> Apr 15 22:01:40 xencluster1 crmd: [8096]: ERROR: process_lrm_event:
>>>> LRM
>>>>>>>> operation clvm-xenvg:0_monitor_10000 (92) Timed Out
>>>>>>>> (timeout=30000ms)
>>>>>>>> Apr 15 22:01:42 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ac
>>>>>>>> Apr 15 22:01:42 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ab
>>>>>>>> Apr 15 22:01:42 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ac
>>>>>>>> Apr 15 22:01:42 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ab
>>>>>>>>
>>>>>>>>
>>>>>>>> pr 15 22:01:50 xencluster1 clvmd[8763]: Send local reply
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: Read on local socket 14,
>>>>>>>> len
>>>> =
>>>>>> 32
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: check_all_clvmds_running
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: down_callback. node
>>>> 1226064394,
>>>>>>>> state = 3
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: down_callback. node
>>>> 1309950474,
>>>>>>>> state = 3
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: Got pre command
>>>>>>>> condition...
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: Writing status 0 down pipe
>>>>>>>> 16
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: Waiting to do post command
>>>>>>>> -
>>>>>>>> state = 0
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: read on PIPE 15: 4 bytes:
>>>>>> status:
>>>>>>>> 0
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: background routine status
>>>>>>>> was
>>>>>>>> 0, sock_client=0x7f5c40057c10
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: distribute command: XID =
>>>> 15473
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: num_nodes = 2
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: add_to_lvmqueue:
>>>>>>>> cmd=0x7f5c4000dbd0. client=0x7f5c40057c10, msg=0x7f5c40057880,
>>>>>>>> len=32,
>>>>>>>> csid=(nil), xid=15473
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: Sending message to all
>>>> cluster
>>>>>>>> nodes
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: process_work_item: local
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: process_local_command:
>>>> unknown
>>>>>>>> (0x2d) msg=0x7f5c40010e80, msglen =32, client=0x7f5c40057c10
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: Syncing device names
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: Reply from node 4914420a: 0
>>>>>> bytes
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: Got 1 replies, expecting: 2
>>>>>>>> Apr 15 22:01:50 xencluster1 clvmd[8763]: LVM thread waiting for work
>>>>>>>> Apr 15 22:01:50 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ab
>>>>>>>> Apr 15 22:01:50 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ac
>>>>>>>> Apr 15 22:01:50 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ab
>>>>>>>> ...
>>>>>>>> Apr 15 22:02:19 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ac
>>>>>>>> Apr 15 22:02:20 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ab
>>>>>>>> Apr 15 22:02:20 xencluster1 lrmd: [8093]: WARN: clvm-xenvg:0:monitor
>>>>>>>> process (PID 771) timed out (try 1). Killing with signal SIGTERM
>>>> (15).
>>>>>>>> Apr 15 22:02:20 xencluster1 lrmd: [8093]: WARN: operation
>>>>>>>> monitor[92]
>>>> on
>>>>>>>> clvm-xenvg:0 for client 8096: pid 771 timed out
>>>>>>>> Apr 15 22:02:20 xencluster1 crmd: [8096]: ERROR: process_lrm_event:
>>>> LRM
>>>>>>>> operation clvm-xenvg:0_monitor_10000 (92) Timed Out
>>>>>>>> (timeout=30000ms)
>>>>>>>> Apr 15 22:02:20 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ac
>>>>>>>> Apr 15 22:02:20 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ab
>>>>>>>> ...
>>>>>>>> Apr 15 22:02:30 xencluster1 clvmd[8763]: Got new connection on fd 17
>>>>>>>> Apr 15 22:02:30 xencluster1 clvmd[8763]: Read on local socket 17,
>>>>>>>> len
>>>> =
>>>>>> 28
>>>>>>>> Apr 15 22:02:30 xencluster1 clvmd[8763]: creating pipe, [18, 19]
>>>>>>>> Apr 15 22:02:30 xencluster1 clvmd[8763]: Creating pre&post thread
>>>>>>>> Apr 15 22:02:30 xencluster1 clvmd[8763]: Created pre&post thread,
>>>> state
>>>>>> = 0
>>>>>>>> Apr 15 22:02:30 xencluster1 clvmd[8763]: in sub thread: client =
>>>>>>>> 0x7f5c40058080
>>>>>>>> Apr 15 22:02:30 xencluster1 clvmd[8763]: Sub thread ready for work.
>>>>>>>> Apr 15 22:02:30 xencluster1 clvmd[8763]: doing PRE command LOCK_VG
>>>>>>>> 'V_XenVG' at 1 (client=0x7f5c40058080)
>>>>>>>> Apr 15 22:02:30 xencluster1 clvmd[8763]: lock_resource 'V_XenVG',
>>>>>>>> flags=0, mode=3
>>>>>>>> Apr 15 22:02:30 xencluster1 clvmd[8763]: lock_resource returning 0,
>>>>>>>> lock_id=3a0001
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Apr 15 22:02:48 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ab
>>>>>>>> Apr 15 22:02:48 xencluster1 cib: [8092]: info: cib_stats: Processed
>>>>>>>> 4
>>>>>>>> operations (2500.00us average, 0% utilization) in the last 10min
>>>>>>>> Apr 15 22:02:48 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ac
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Apr 15 22:03:00 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ac
>>>>>>>> Apr 15 22:03:00 xencluster1 lrmd: [8093]: WARN: clvm-xenvg:0:monitor
>>>>>>>> process (PID 2256) timed out (try 1). Killing with signal SIGTERM
>>>> (15).
>>>>>>>> Apr 15 22:03:00 xencluster1 lrmd: [8093]: WARN: operation
>>>>>>>> monitor[92]
>>>> on
>>>>>>>> clvm-xenvg:0 for client 8096: pid 2256 timed out
>>>>>>>> Apr 15 22:03:00 xencluster1 crmd: [8096]: ERROR: process_lrm_event:
>>>> LRM
>>>>>>>> operation clvm-xenvg:0_monitor_10000 (92) Timed Out
>>>>>>>> (timeout=30000ms)
>>>>>>>> Apr 15 22:03:00 xencluster1 corosync[8071]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 21ab
>>>>>>>>
>>>>>>>>
>>>>>>>> Node2 Log
>>>>>>>>
>>>>>>>> Apr 15 22:00:51 xencluster2 clvmd[12666]: Joined child thread
>>>>>>>> Apr 15 22:00:51 xencluster2 clvmd[12666]: ret == 0, errno = 0.
>>>> removing
>>>>>>>> client
>>>>>>>> Apr 15 22:00:51 xencluster2 clvmd[12666]: add_to_lvmqueue:
>>>>>>>> cmd=0x7fa3d0028710. client=0x7fa3d0028760, msg=(nil), len=0,
>>>> csid=(nil),
>>>>>>>> xid=14778
>>>>>>>> Apr 15 22:00:51 xencluster2 clvmd[12666]: process_work_item: free fd
>>>> -1
>>>>>>>> Apr 15 22:00:51 xencluster2 clvmd[12666]: LVM thread waiting for
>>>>>>>> work
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: crm_timer_popped:
>>>>>>>> PEngine Recheck Timer (I_PE_CALC) just popped (900000ms)
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: do_state_transition:
>>>>>>>> State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC
>>>>>>>> cause=C_TIMER_POPPED origin=crm_timer_popped ]
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: do_state_transition:
>>>>>>>> Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: do_state_transition:
>>>> All
>>>>>>>> 2 cluster nodes are eligible to run resources.
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: do_pe_invoke: Query
>>>> 983:
>>>>>>>> Requesting the current CIB: S_POLICY_ENGINE
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info:
>>>>>>>> do_pe_invoke_callback:
>>>>>>>> Invoking the PE: query=983, ref=pe_calc-dc-1334520052-604, seq=2212,
>>>>>>>> quorate=1
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: unpack_config:
>>>>>>>> On
>>>>>>>> loss of CCM Quorum: Ignore
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation clvm:1_last_failure_0 found resource clvm:1 active on
>>>>>> xencluster2
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation STONITH-Node2_last_failure_0 found resource STONITH-Node2
>>>>>>>> active on xencluster2
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation STONITH--Node1_last_failure_0 found resource
>>>>>>>> STONITH--Node1
>>>>>>>> active on xencluster2
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation cmirror:1_last_failure_0 found resource cmirror:1 active
>>>>>>>> on
>>>>>>>> xencluster2
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation dlm:1_last_failure_0 found resource dlm:1 active on
>>>>>> xencluster2
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation o2cb:1_last_failure_0 found resource o2cb:1 active on
>>>>>> xencluster2
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation dlm:0_last_failure_0 found resource dlm:0 active on
>>>>>> xencluster1
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation clvm:0_last_failure_0 found resource clvm:0 active on
>>>>>> xencluster1
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation o2cb:0_last_failure_0 found resource o2cb:0 active on
>>>>>> xencluster1
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation clvm-xenvg:0_last_failure_0 found resource clvm-xenvg:0
>>>> active
>>>>>>>> on xencluster1
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: WARN: unpack_rsc_op:
>>>>>>>> Processing failed op vm-deprepo_last_failure_0 on xencluster1:
>>>>>>>> unknown
>>>>>>>> error (1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> STONITH-Node2 (Started xencluster2)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> STONITH--Node1 (Started xencluster2)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> dlm:0 (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> clvm:0 (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> o2cb:0 (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> cmirror:0 (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> dlm:1 (Started xencluster2)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> clvm:1 (Started xencluster2)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> o2cb:1 (Started xencluster2)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> cmirror:1 (Started xencluster2)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> clvm-xenvg:0 (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> fs-config-xen:0 (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> clvm-xenvg:1 (Started xencluster2)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> fs-config-xen:1 (Started xencluster2)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-ilmt (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-topdesk-entw (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-otbridge (Started xencluster2)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-virenscanner (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-deprepo (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-traumschiff (Started xencluster2)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-cim (Started xencluster2)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-oneiroi (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-intranet (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-soa (Started xencluster1)
>>>>>>>> Apr 15 22:00:52 xencluster2 pengine: [7677]: notice:
>>>> process_pe_message:
>>>>>>>> Transition 80: PEngine Input stored in:
>>>>>> /var/lib/pengine/pe-input-271.bz2
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: do_state_transition:
>>>>>>>> State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [
>>>>>>>> input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: unpack_graph:
>>>>>>>> Unpacked
>>>>>>>> transition 80: 0 actions in 0 synapses
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: do_te_invoke:
>>>> Processing
>>>>>>>> graph 80 (ref=pe_calc-dc-1334520052-604) derived from
>>>>>>>> /var/lib/pengine/pe-input-271.bz2
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: run_graph:
>>>>>>>> ====================================================
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: notice: run_graph:
>>>> Transition
>>>>>>>> 80 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
>>>>>>>> Source=/var/lib/pengine/pe-input-271.bz2): Complete
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: te_graph_trigger:
>>>>>>>> Transition 80 is now complete
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: notify_crmd:
>>>> Transition
>>>>>>>> 80 status: done - <null>
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: do_state_transition:
>>>>>>>> State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
>>>>>>>> cause=C_FSA_INTERNAL origin=notify_crmd ]
>>>>>>>> Apr 15 22:00:52 xencluster2 crmd: [7678]: info: do_state_transition:
>>>>>>>> Starting PEngine Recheck Timer
>>>>>>>> Apr 15 22:01:00 xencluster2 clvmd[12666]: 1309950474 got message
>>>>>>>> from
>>>>>>>> nodeid 1226064394 for 0. len 32
>>>>>>>> Apr 15 22:01:00 xencluster2 clvmd[12666]: add_to_lvmqueue:
>>>>>>>> cmd=0x7fa3d0028780. client=0x6934a0, msg=0x7fa3d910063c, len=32,
>>>>>>>> csid=0x7fffb4e5d944, xid=0
>>>>>>>> Apr 15 22:01:00 xencluster2 clvmd[12666]: process_work_item: remote
>>>>>>>> Apr 15 22:01:00 xencluster2 clvmd[12666]: process_remote_command
>>>> unknown
>>>>>>>> (0x2d) for clientid 0x5000000 XID 15337 on node 4914420a
>>>>>>>> Apr 15 22:01:00 xencluster2 clvmd[12666]: Syncing device names
>>>>>>>> Apr 15 22:01:00 xencluster2 clvmd[12666]: LVM thread waiting for
>>>>>>>> work
>>>>>>>> Apr 15 22:01:00 xencluster2 clvmd[12666]: 1309950474 got message
>>>>>>>> from
>>>>>>>> nodeid 1309950474 for 1226064394. len 18
>>>>>>>> Apr 15 22:01:00 xencluster2 clvmd[12666]: 1309950474 got message
>>>>>>>> from
>>>>>>>> nodeid 1226064394 for 0. len 32
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Apr 15 22:01:02 xencluster2 clvmd[12666]: Joined child thread
>>>>>>>> Apr 15 22:01:02 xencluster2 clvmd[12666]: ret == 0, errno = 0.
>>>> removing
>>>>>>>> client
>>>>>>>> Apr 15 22:01:02 xencluster2 clvmd[12666]: add_to_lvmqueue:
>>>>>>>> cmd=0x7fa3d004d7a0. client=0x7fa3d004d7f0, msg=(nil), len=0,
>>>> csid=(nil),
>>>>>>>> xid=14848
>>>>>>>> Apr 15 22:01:02 xencluster2 clvmd[12666]: process_work_item: free fd
>>>> -1
>>>>>>>> Apr 15 22:01:02 xencluster2 clvmd[12666]: LVM thread waiting for
>>>>>>>> work
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_command:
>>>>>>>> Processed st_execute from lrmd: rc=-1
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith: parse config info
>>>> info=xencluster1
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith: meatware device OK.
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH--Node1: Performing: stonith -t meatware -S
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH--Node1: success: 0
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_device_execute: Nothing to do for STONITH--Node1
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_command:
>>>>>>>> Processed st_execute from lrmd: rc=-1
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith: parse config info
>>>> info=xencluster2
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith: meatware device OK.
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH-Node2: Performing: stonith -t meatware -S
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH-Node2: success: 0
>>>>>>>> Apr 15 22:01:06 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_device_execute: Nothing to do for STONITH-Node2
>>>>>>>> Apr 15 22:01:10 xencluster2 clvmd[12666]: 1309950474 got message
>>>>>>>> from
>>>>>>>> nodeid 1226064394 for 0. len 32
>>>>>>>> Apr 15 22:01:10 xencluster2 clvmd[12666]: add_to_lvmqueue:
>>>>>>>> cmd=0x7fa3d004d810. client=0x6934a0, msg=0x7fa3d9100cfc, len=32,
>>>>>>>> csid=0x7fffb4e5d944, xid=0
>>>>>>>> Apr 15 22:01:10 xencluster2 clvmd[12666]: process_work_item: remote
>>>>>>>> Apr 15 22:01:10 xencluster2 clvmd[12666]: process_remote_command
>>>> unknown
>>>>>>>> (0x2d) for clientid 0x5000000 XID 15407 on node 4914420a
>>>>>>>> Apr 15 22:01:10 xencluster2 clvmd[12666]: Syncing device names
>>>>>>>> Apr 15 22:01:10 xencluster2 clvmd[12666]: LVM thread waiting for
>>>>>>>> work
>>>>>>>> Apr 15 22:01:10 xencluster2 clvmd[12666]: 1309950474 got message
>>>>>>>> from
>>>>>>>> nodeid 1309950474 for 1226064394. len 18
>>>>>>>> Apr 15 22:01:10 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196
>>>>>>>> Apr 15 22:01:10 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196
>>>>>>>> Apr 15 22:01:10 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196
>>>>>>>> ....
>>>>>>>> Apr 15 22:01:11 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196
>>>>>>>> Apr 15 22:01:12 xencluster2 clvmd[12666]: Got new connection on fd 5
>>>>>>>> Apr 15 22:01:12 xencluster2 clvmd[12666]: Read on local socket 5,
>>>>>>>> len
>>>> =
>>>>>> 28
>>>>>>>> Apr 15 22:01:12 xencluster2 clvmd[12666]: creating pipe, [12, 13]
>>>>>>>> Apr 15 22:01:12 xencluster2 clvmd[12666]: Creating pre&post thread
>>>>>>>> Apr 15 22:01:12 xencluster2 clvmd[12666]: Created pre&post thread,
>>>>>> state =
>>>>>>>> 0
>>>>>>>> Apr 15 22:01:12 xencluster2 clvmd[12666]: in sub thread: client =
>>>>>>>> 0x7fa3d004d810
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Apr 15 22:01:12 xencluster2 clvmd[12666]: Syncing device names
>>>>>>>> Apr 15 22:01:12 xencluster2 clvmd[12666]: Reply from node 4e14420a:
>>>>>>>> 0
>>>>>> bytes
>>>>>>>> Apr 15 22:01:12 xencluster2 clvmd[12666]: Got 1 replies, expecting:
>>>>>>>> 2
>>>>>>>> Apr 15 22:01:12 xencluster2 clvmd[12666]: LVM thread waiting for
>>>>>>>> work
>>>>>>>> Apr 15 22:01:12 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196
>>>>>>>> Apr 15 22:01:12 xencluster2 clvmd[12666]: 1309950474 got message
>>>>>>>> from
>>>>>>>> nodeid 1309950474 for 0. len 32
>>>>>>>> Apr 15 22:01:12 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196
>>>>>>>> Apr 15 22:01:12 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196
>>>>>>>> ....
>>>>>>>> Apr 15 22:01:21 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:21 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_command:
>>>>>>>> Processed st_execute from lrmd: rc=-1
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith: parse config info
>>>> info=xencluster1
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith: meatware device OK.
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH--Node1: Performing: stonith -t meatware -S
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH--Node1: success: 0
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_device_execute: Nothing to do for STONITH--Node1
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_command:
>>>>>>>> Processed st_execute from lrmd: rc=-1
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith: parse config info
>>>> info=xencluster2
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith: meatware device OK.
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH-Node2: Performing: stonith -t meatware -S
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH-Node2: success: 0
>>>>>>>> Apr 15 22:01:21 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_device_execute: Nothing to do for STONITH-Node2
>>>>>>>> Apr 15 22:01:21 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:22 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:22 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> ....
>>>>>>>> Apr 15 22:01:36 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:36 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_command:
>>>>>>>> Processed st_execute from lrmd: rc=-1
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith: parse config info
>>>> info=xencluster1
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith: meatware device OK.
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH--Node1: Performing: stonith -t meatware -S
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH--Node1: success: 0
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_device_execute: Nothing to do for STONITH--Node1
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_command:
>>>>>>>> Processed st_execute from lrmd: rc=-1
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith: parse config info
>>>> info=xencluster2
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith: meatware device OK.
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH-Node2: Performing: stonith -t meatware -S
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith-ng: [8283]: info: log_operation:
>>>>>>>> STONITH-Node2: success: 0
>>>>>>>> Apr 15 22:01:36 xencluster2 stonith-ng: [8283]: info:
>>>>>>>> stonith_device_execute: Nothing to do for STONITH-Node2
>>>>>>>> Apr 15 22:01:36 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:37 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> ....
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: process_graph_event:
>>>>>>>> Detected action clvm-xenvg:0_monitor_10000 from a different
>>>> transition:
>>>>>>>> 62 vs. 80
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info:
>>>> abort_transition_graph:
>>>>>>>> process_graph_event:476 - Triggered transition abort (complete=1,
>>>>>>>> tag=lrm_rsc_op, id=clvm-xenvg:0_monitor_10000,
>>>>>>>> magic=2:-2;61:62:0:72497771-fdb2-4ca7-a723-2cc360861bb0,
>>>> cib=0.5816.4) :
>>>>>>>> Old event
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: WARN: update_failcount:
>>>>>>>> Updating failcount for clvm-xenvg:0 on xencluster1 after failed
>>>> monitor:
>>>>>>>> rc=-2 (update=value++, time=1334520100)
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: do_state_transition:
>>>>>>>> State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC
>>>>>>>> cause=C_FSA_INTERNAL origin=abort_transition_graph ]
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: do_state_transition:
>>>> All
>>>>>>>> 2 cluster nodes are eligible to run resources.
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: do_pe_invoke: Query
>>>> 984:
>>>>>>>> Requesting the current CIB: S_POLICY_ENGINE
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d
>>>>>>>> ...
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info:
>>>>>>>> do_pe_invoke_callback:
>>>>>>>> Invoking the PE: query=984, ref=pe_calc-dc-1334520100-605, seq=2212,
>>>>>>>> quorate=1
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d
>>>>>>>> ...
>>>>>>>>
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_config:
>>>>>>>> On
>>>>>>>> loss of CCM Quorum: Ignore
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation clvm:1_last_failure_0 found resource clvm:1 active on
>>>>>> xencluster2
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation STONITH-Node2_last_failure_0 found resource STONITH-Node2
>>>>>>>> active on xencluster2
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation STONITH--Node1_last_failure_0 found resource
>>>>>>>> STONITH--Node1
>>>>>>>> active on xencluster2
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation cmirror:1_last_failure_0 found resource cmirror:1 active
>>>>>>>> on
>>>>>>>> xencluster2
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation dlm:1_last_failure_0 found resource dlm:1 active on
>>>>>> xencluster2
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation o2cb:1_last_failure_0 found resource o2cb:1 active on
>>>>>> xencluster2
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation cmirror:0_last_failure_0 found resource cmirror:0 active
>>>>>>>> on
>>>>>>>> xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation dlm:0_last_failure_0 found resource dlm:0 active on
>>>>>> xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation clvm:0_last_failure_0 found resource clvm:0 active on
>>>>>> xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation o2cb:0_last_failure_0 found resource o2cb:0 active on
>>>>>> xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: unpack_rsc_op:
>>>>>>>> Operation clvm-xenvg:0_last_failure_0 found resource clvm-xenvg:0
>>>> active
>>>>>>>> on xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: WARN: unpack_rsc_op:
>>>>>>>> Processing failed op clvm-xenvg:0_monitor_10000 on xencluster1:
>>>> unknown
>>>>>>>> exec error (-2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: WARN: unpack_rsc_op:
>>>>>>>> Processing failed op vm-deprepo_last_failure_0 on xencluster1:
>>>>>>>> unknown
>>>>>>>> error (1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: RecurringOp:
>>>> Start
>>>>>>>> recurring monitor (10s) for clvm-xenvg:0 on xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> STONITH-Node2 (Started xencluster2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> STONITH--Node1 (Started xencluster2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> dlm:0 (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> clvm:0 (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> o2cb:0 (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> cmirror:0 (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> dlm:1 (Started xencluster2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> clvm:1 (Started xencluster2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> o2cb:1 (Started xencluster2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> cmirror:1 (Started xencluster2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>> Recover
>>>>>>>> clvm-xenvg:0 (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>> Restart
>>>>>>>> fs-config-xen:0 (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> clvm-xenvg:1 (Started xencluster2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> fs-config-xen:1 (Started xencluster2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>> Restart
>>>>>>>> vm-ilmt (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>> Restart
>>>>>>>> vm-topdesk-entw (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-otbridge (Started xencluster2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>> Restart
>>>>>>>> vm-virenscanner (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>> Restart
>>>>>>>> vm-deprepo (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-traumschiff (Started xencluster2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>>>>>> Leave
>>>>>>>> vm-cim (Started xencluster2)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>> Restart
>>>>>>>> vm-oneiroi (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>> Restart
>>>>>>>> vm-intranet (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice: LogActions:
>>>> Restart
>>>>>>>> vm-soa (Started xencluster1)
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: do_state_transition:
>>>>>>>> State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [
>>>>>>>> input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: unpack_graph:
>>>>>>>> Unpacked
>>>>>>>> transition 81: 39 actions in 39 synapses
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: do_te_invoke:
>>>> Processing
>>>>>>>> graph 81 (ref=pe_calc-dc-1334520100-605) derived from
>>>>>>>> /var/lib/pengine/pe-input-272.bz2
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: te_rsc_command:
>>>>>>>> Initiating action 82: stop vm-ilmt_stop_0 on xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: te_rsc_command:
>>>>>>>> Initiating action 86: stop vm-topdesk-entw_stop_0 on xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: te_rsc_command:
>>>>>>>> Initiating action 90: stop vm-virenscanner_stop_0 on xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: te_rsc_command:
>>>>>>>> Initiating action 92: stop vm-deprepo_stop_0 on xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: te_rsc_command:
>>>>>>>> Initiating action 98: stop vm-oneiroi_stop_0 on xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: te_rsc_command:
>>>>>>>> Initiating action 100: stop vm-intranet_stop_0 on xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 crmd: [7678]: info: te_rsc_command:
>>>>>>>> Initiating action 102: stop vm-soa_stop_0 on xencluster1
>>>>>>>> Apr 15 22:01:40 xencluster2 pengine: [7677]: notice:
>>>> process_pe_message:
>>>>>>>> Transition 81: PEngine Input stored in:
>>>>>> /var/lib/pengine/pe-input-272.bz2
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d 219e 219f 21a0 21a1 21a2 21a3 21a4
>>>>>>>> 21a5
>>>>>>>> Apr 15 22:01:40 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d 219e 219f 21a0 21a1 21a2 21a3 21a4
>>>>>>>> 21a5
>>>>>>>> ...
>>>>>>>> Apr 15 22:01:41 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d 219e 219f 21a0 21a1 21a2 21a3 21a4
>>>>>>>> 21a5
>>>>>>>> Apr 15 22:01:42 xencluster2 lrmd: [7675]: WARN: clvm-xenvg:1:monitor
>>>>>>>> process (PID 29785) timed out (try 1). Killing with signal SIGTERM
>>>>>> (15).
>>>>>>>> Apr 15 22:01:42 xencluster2 lrmd: [7675]: WARN: operation
>>>>>>>> monitor[125]
>>>>>>>> on clvm-xenvg:1 for client 7678: pid 29785 timed out
>>>>>>>> Apr 15 22:01:42 xencluster2 crmd: [7678]: ERROR: process_lrm_event:
>>>> LRM
>>>>>>>> operation clvm-xenvg:1_monitor_10000 (125) Timed Out
>>>>>>>> (timeout=30000ms)
>>>>>>>> Apr 15 22:01:42 xencluster2 crmd: [7678]: info: process_graph_event:
>>>>>>>> Detected action clvm-xenvg:1_monitor_10000 from a different
>>>> transition:
>>>>>>>> 68 vs. 81
>>>>>>>> Apr 15 22:01:42 xencluster2 crmd: [7678]: info:
>>>> abort_transition_graph:
>>>>>>>> process_graph_event:476 - Triggered transition abort (complete=0,
>>>>>>>> tag=lrm_rsc_op, id=clvm-xenvg:1_monitor_10000,
>>>>>>>> magic=2:-2;61:68:0:72497771-fdb2-4ca7-a723-2cc360861bb0,
>>>> cib=0.5816.5) :
>>>>>>>> Old event
>>>>>>>> Apr 15 22:01:42 xencluster2 crmd: [7678]: info:
>>>>>>>> update_abort_priority:
>>>>>>>> Abort priority upgraded from 0 to 1000000
>>>>>>>> Apr 15 22:01:42 xencluster2 crmd: [7678]: info:
>>>>>>>> update_abort_priority:
>>>>>>>> Abort action done superceeded by restart
>>>>>>>> Apr 15 22:01:42 xencluster2 crmd: [7678]: WARN: update_failcount:
>>>>>>>> Updating failcount for clvm-xenvg:1 on xencluster2 after failed
>>>> monitor:
>>>>>>>> rc=-2 (update=value++, time=1334520102)
>>>>>>>> Apr 15 22:01:42 xencluster2 attrd: [7676]: notice:
>>>> attrd_trigger_update:
>>>>>>>> Sending flush op to all hosts for: fail-count-clvm-xenvg:1 (1)
>>>>>>>> Apr 15 22:01:42 xencluster2 attrd: [7676]: notice:
>>>> attrd_perform_update:
>>>>>>>> Sent update 448: fail-count-clvm-xenvg:1=1
>>>>>>>> Apr 15 22:01:42 xencluster2 attrd: [7676]: notice:
>>>> attrd_trigger_update:
>>>>>>>> Sending flush op to all hosts for: last-failure-clvm-xenvg:1
>>>>>> (1334520102)
>>>>>>>> Apr 15 22:01:42 xencluster2 crmd: [7678]: info:
>>>> abort_transition_graph:
>>>>>>>> te_update_diff:176 - Triggered transition abort (complete=0,
>>>> tag=nvpair,
>>>>>>>> id=status-xencluster2-fail-count-clvm-xenvg.1,
>>>>>>>> name=fail-count-clvm-xenvg:1, value=1, magic=NA, cib=0.5816.6) :
>>>>>>>> Transient attribute: update
>>>>>>>> Apr 15 22:01:42 xencluster2 attrd: [7676]: notice:
>>>> attrd_perform_update:
>>>>>>>> Sent update 451: last-failure-clvm-xenvg:1=1334520102
>>>>>>>> Apr 15 22:01:42 xencluster2 crmd: [7678]: info:
>>>> abort_transition_graph:
>>>>>>>> te_update_diff:176 - Triggered transition abort (complete=0,
>>>> tag=nvpair,
>>>>>>>> id=status-xencluster2-last-failure-clvm-xenvg.1,
>>>>>>>> name=last-failure-clvm-xenvg:1, value=1334520102, magic=NA,
>>>>>>>> cib=0.5816.7) : Transient attribute: update
>>>>>>>> Apr 15 22:01:42 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d 219e 219f 21a0 21a1 21a2 21a3 21a4
>>>>>>>> 21a5
>>>>>>>> Apr 15 22:01:42 xencluster2 corosync[7666]: [TOTEM ] Retransmit
>>>> List:
>>>>>>>> 2196 2197 219a 219b 219c 219d 219e 219f 21a0 21a1 21a2 21a3 21a4
>>>>>>>> 21a5
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mit freundlichen Gr??en
>>>>>>>>
>>>>>>>> Frank Meier
>>>>>>>> UNIX-Basis
>>>>>>>>
>>>>>>>> Hamm Reno Group GmbH
>>>>>>>> Industriegebiet West | D-66987 Thaleischweiler-Fr?schen
>>>>>>>> T.+49(0)6334 444-8322 | F.+49(0)6334 444-8190
>>>>>>>> frank.meier at hr-group.de | www.reno.de
>>>>>>>> ___________________________________________________________________
>>>>>>>>
>>>>>>>> Sitz: Am Tie 7 | D-49086 Osnabr?ck
>>>>>>>> Handelsregister Osnabr?ck HRB 19587
>>>>>>>> Gesch?ftsf?hrer: Hans-J?rgen de Fries,
>>>>>>>> Jens Gransee, Manfred Klumpp,
>>>>>>>> Robert Reisch
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>>>
>>>>>>>> Project Home: http://www.clusterlabs.org
>>>>>>>> Getting started:
>>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>> -------------- next part --------------
>>>>>>> An HTML attachment was scrubbed...
>>>>>>> URL: <
>>>>>>
>>>> http://oss.clusterlabs.org/pipermail/pacemaker/attachments/20120419/00a36dbe/attachment.html
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Pacemaker mailing list
>>>>>>> Pacemaker at oss.clusterlabs.org
>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>>
>>>>>>>
>>>>>>> End of Pacemaker Digest, Vol 53, Issue 38
>>>>>>> *****************************************
>>>>>>
>>>>>> _______________________________________________
>>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>>
>>>>>> Project Home: http://www.clusterlabs.org
>>>>>> Getting started:
>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>>> Bugs: http://bugs.clusterlabs.org
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>> -------------- next part --------------
>>>>> An HTML attachment was scrubbed...
>>>>> URL: <
>>>> http://oss.clusterlabs.org/pipermail/pacemaker/attachments/20120419/63b888a3/attachment.html
>>>>>
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> _______________________________________________
>>>>> Pacemaker mailing list
>>>>> Pacemaker at oss.clusterlabs.org
>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>
>>>>>
>>>>> End of Pacemaker Digest, Vol 53, Issue 40
>>>>> *****************************************
>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://bugs.clusterlabs.org
>>>>
>>>
>>>
>>>
>>> --
>>> esta es mi vida e me la vivo hasta que dios quiera
>>> -------------- next part --------------
>>> An HTML attachment was scrubbed...
>>> URL:
>>> <http://oss.clusterlabs.org/pipermail/pacemaker/attachments/20120419/f21bf7c7/attachment.html>
>>>
>>> ------------------------------
>>>
>>> _______________________________________________
>>> Pacemaker mailing list
>>> Pacemaker at oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>>
>>> End of Pacemaker Digest, Vol 53, Issue 42
>>> *****************************************
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>>
>>
>>
>> --
>> esta es mi vida e me la vivo hasta que dios quiera
>>
>>
>>
>> ------------------------------
>>
>> _______________________________________________
>> Pacemaker mailing list
>> Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>>
>> End of Pacemaker Digest, Vol 53, Issue 63
>> *****************************************
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
--
esta es mi vida e me la vivo hasta que dios quiera
More information about the Pacemaker
mailing list