[Pacemaker] ERROR: te_graph_trigger: Transition failed: terminated pacemaker's problem or mine?
Lars Ellenberg
lars.ellenberg at linbit.com
Fri Apr 27 15:40:57 EDT 2012
On Fri, Apr 27, 2012 at 11:31:23AM +0100, Tim Small wrote:
> Hi,
>
> I'm trying to get to the bottom of a problem I'm seeing with a cluster.
> At this stage I'm unclear as to whether the issue is with the config or
> not - the generated error messages seem unclear. So I'm not sure
> whether I should be staring at the config or the source code at this
> point, and would appreciate a clue!
>
> I'm running with some of the (live) resources in an unmanaged state
> whilst testing fail-over with other (non-dependant) resources.
>
> The managed resources are a number of OpenVZ virtual machines (each
> comprising 3 primitives - file-system + OpenVZ VE + SendArp). The
> filesystems are on LVM volume groups, and the single LVM PV for each
> volume group resides on a DRBD volume. There are n virtual machines per
> DRBD volume.
>
> I'm running pacemaker 1.0.9.1+hg15626-1 on Debian 6.0. Here are some of
> the messages (configuration follows at the end of the email):
Upgrading to 1.0.12, or 1.1.7, may get you a little further.
It would not solve the "I need to stop that resource first, but I can
not as it is unmanaged" dependency problem you apparently have here.
I think you simply have some copy'n'paste errors in your constraints,
calypso should be ordered with essex03, not 02.
May not be the only problem, though.
BTW, LCMC, respectively the cluster resource and constraint graph view
it presents to you, can help to "just see" this kind of error.
Some more comments inline.
> Apr 27 11:06:35 fig crmd: [395]: info: crm_timer_popped: PEngine Recheck Timer (I_PE_CALC) just popped!
> Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]
> Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED
> Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
> Apr 27 11:06:35 fig crmd: [395]: info: do_pe_invoke: Query 985: Requesting the current CIB: S_POLICY_ENGINE
> Apr 27 11:06:35 fig crmd: [395]: info: do_pe_invoke_callback: Invoking the PE: query=985, ref=pe_calc-dc-1335521195-1437, seq=184, quorate=1
> Apr 27 11:06:35 fig pengine: [394]: notice: unpack_config: On loss of CCM Quorum: Ignore
> Apr 27 11:06:35 fig pengine: [394]: info: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Apr 27 11:06:35 fig pengine: [394]: info: determine_online_status: Node fig is online
> Apr 27 11:06:35 fig pengine: [394]: info: determine_online_status: Node hazel is online
> Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource essex03-LVM isnt managed
> Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource calypso-FS isnt managed
> Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource calypso-VE isnt managed
> Apr 27 11:06:35 fig pengine: [394]: notice: unpack_rsc_op: Operation essex03-DRBD:0_monitor_0 found resource essex03-DRBD:0 active in master mode on hazel
> Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource essex03-DRBD:0 isnt managed
> Apr 27 11:06:35 fig pengine: [394]: notice: unpack_rsc_op: Operation essex02-DRBD:1_monitor_0 found resource essex02-DRBD:1 active in master mode on hazel
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: artemis-FS#011(ocf::heartbeat:Filesystem):#011Stopped
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: artemis-SendArp#011(ocf::heartbeat:SendArp):#011Stopped
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: artemis-VE#011(ocf::heartbeat:ManageVE):#011Stopped (unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: athena-FS#011(ocf::heartbeat:Filesystem):#011Stopped
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: athena-SendArp#011(ocf::heartbeat:SendArp):#011Stopped
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: athena-VE#011(ocf::heartbeat:ManageVE):#011Stopped (unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: calypso-FS#011(ocf::heartbeat:Filesystem):#011Started hazel (unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: calypso-SendArp#011(ocf::heartbeat:SendArp):#011Stopped
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: calypso-VE#011(ocf::heartbeat:ManageVE):#011Started hazel (unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: epione-FS#011(ocf::heartbeat:Filesystem):#011Stopped
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: epione-SendArp#011(ocf::heartbeat:SendArp):#011Stopped
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: epione-VE#011(ocf::heartbeat:ManageVE):#011Stopped (unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essex02-LVM#011(ocf::heartbeat:LVM):#011Started hazel
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essex03-LVM#011(ocf::heartbeat:LVM):#011Started hazel (unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essextest-FS#011(ocf::heartbeat:Filesystem):#011Stopped
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essextest-SendArp#011(ocf::heartbeat:SendArp):#011Stopped
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essextest-VE#011(ocf::heartbeat:ManageVE):#011Stopped
> Apr 27 11:06:35 fig pengine: [394]: notice: clone_print: Master/Slave Set: ms-drbd-essex02
> Apr 27 11:06:35 fig pengine: [394]: notice: short_print: Masters: [ hazel ]
> Apr 27 11:06:35 fig pengine: [394]: notice: short_print: Slaves: [ fig ]
> Apr 27 11:06:35 fig pengine: [394]: notice: clone_print: Master/Slave Set: ms-drbd-essex03 (unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essex03-DRBD:0#011(ocf::linbit:drbd):#011Master hazel (unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: short_print: Stopped: [ essex03-DRBD:1 ]
> Apr 27 11:06:35 fig pengine: [394]: info: get_failcount: ms-drbd-essex03 has failed 1 times on hazel
> Apr 27 11:06:35 fig pengine: [394]: notice: common_apply_stickiness: ms-drbd-essex03 can fail 999999 more times on hazel before being forced off
> Apr 27 11:06:35 fig pengine: [394]: info: get_failcount: ms-drbd-essex03 has failed 1 times on hazel
> Apr 27 11:06:35 fig pengine: [394]: notice: common_apply_stickiness: ms-drbd-essex03 can fail 999999 more times on hazel before being forced off
> Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting essex02-DRBD:0 (Slave fig)
> Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex02: Promoted 1 instances of a possible 1 to master
> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource artemis-VE allocated to 'nowhere': inactive
> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Resource artemis-SendArp cannot run anywhere
> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource athena-VE allocated to 'nowhere': inactive
> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Resource athena-SendArp cannot run anywhere
> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource essex03-DRBD:0 allocated to hazel: master
> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource essex03-DRBD:1 allocated to 'nowhere': inactive
> Apr 27 11:06:35 fig pengine: [394]: notice: can_be_master: Forcing unmanaged master essex03-DRBD:0 to remain promoted on hazel
> Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting essex03-DRBD:0 (Master hazel)
> Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex03: Promoted 1 instances of a possible 1 to master
> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource essex03-LVM allocated to hazel: active
> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource calypso-FS allocated to hazel: active
> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource calypso-VE allocated to hazel: active
> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource epione-VE allocated to 'nowhere': inactive
> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Resource epione-SendArp cannot run anywhere
> Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting essex02-DRBD:0 (Slave fig)
> Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex02: Promoted 1 instances of a possible 1 to master
> Apr 27 11:06:35 fig pengine: [394]: notice: can_be_master: Forcing unmanaged master essex03-DRBD:0 to remain promoted on hazel
> Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting essex03-DRBD:0 (Master hazel)
> Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex03: Promoted 1 instances of a possible 1 to master
> Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring monitor (29s) for essex02-DRBD:0 on fig
> Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring monitor (31s) for essex02-DRBD:1 on hazel
> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex02
> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex02
> Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring monitor (29s) for essex02-DRBD:0 on fig
> Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring monitor (31s) for essex02-DRBD:1 on hazel
> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex02
> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex02
> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex03
> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex03
> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex03
> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex03
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start artemis-FS#011(fig)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource artemis-SendArp#011(Stopped)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource artemis-VE#011(Stopped unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start athena-FS#011(fig)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource athena-SendArp#011(Stopped)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource athena-VE#011(Stopped unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource calypso-FS#011(Started unmanaged)
Just one of the things that go wrong below: calypso-FS is Started,
but unmanaged.
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start calypso-SendArp#011(hazel)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource calypso-VE#011(Started unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start epione-FS#011(fig)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource epione-SendArp#011(Stopped)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource epione-VE#011(Stopped unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Move resource essex02-LVM#011(Started hazel -> fig)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource essex03-LVM#011(Started unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start essextest-FS#011(fig)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start essextest-SendArp#011(fig)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start essextest-VE#011(fig)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Promote essex02-DRBD:0#011(Slave -> Master fig)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Demote essex02-DRBD:1#011(Master -> Slave hazel)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource essex03-DRBD:0#011(Master unmanaged)
> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource essex03-DRBD:1#011(Stopped unmanaged)
> Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
> Apr 27 11:06:35 fig crmd: [395]: info: unpack_graph: Unpacked transition 121: 34 actions in 34 synapses
> Apr 27 11:06:35 fig crmd: [395]: info: do_te_invoke: Processing graph 121 (ref=pe_calc-dc-1335521195-1437) derived from /var/lib/pengine/pe-input-82.bz2
> Apr 27 11:06:35 fig crmd: [395]: info: te_pseudo_action: Pseudo action 53 fired and confirmed
> Apr 27 11:06:35 fig pengine: [394]: info: process_pe_message: Transition 121: PEngine Input stored in: /var/lib/pengine/pe-input-82.bz2
> Apr 27 11:06:35 fig crmd: [395]: info: te_rsc_command: Initiating action 119: notify essex02-DRBD:0_pre_notify_demote_0 on fig (local)
> Apr 27 11:06:35 fig crmd: [395]: info: do_lrm_rsc_op: Performing key=119:121:0:3dcf0e40-239f-4eed-9764-9c29003fa305 op=essex02-DRBD:0_notify_0 )
> Apr 27 11:06:35 fig lrmd: [392]: info: rsc:essex02-DRBD:0:594: notify
> Apr 27 11:06:35 fig crmd: [395]: info: te_rsc_command: Initiating action 121: notify essex02-DRBD:1_pre_notify_demote_0 on hazel
> Apr 27 11:06:36 fig crmd: [395]: info: match_graph_event: Action essex02-DRBD:1_pre_notify_demote_0 (121) confirmed on hazel (rc=0)
> Apr 27 11:06:36 fig crmd: [395]: info: process_lrm_event: LRM operation essex02-DRBD:0_notify_0 (call=594, rc=0, cib-update=986, confirmed=true) ok
> Apr 27 11:06:36 fig crmd: [395]: info: match_graph_event: Action essex02-DRBD:0_pre_notify_demote_0 (119) confirmed on fig (rc=0)
> Apr 27 11:06:36 fig crmd: [395]: info: te_pseudo_action: Pseudo action 54 fired and confirmed
> Apr 27 11:06:36 fig crmd: [395]: notice: run_graph: ====================================================
> Apr 27 11:06:36 fig crmd: [395]: WARN: run_graph: Transition 121 (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=30, Source=/var/lib/pengine/pe-input-82.bz2): Terminated
> Apr 27 11:06:36 fig crmd: [395]: ERROR: te_graph_trigger: Transition failed: terminated
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Graph 121 (34 actions in 34 synapses): batch-limit=30 jobs, network-delay=60000ms
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 0 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 6]: Pending (id: artemis-FS_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: Pending (id: essex02-LVM_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 1 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 8]: Pending (id: athena-FS_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: Pending (id: essex02-LVM_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 2 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 12]: Pending (id: calypso-SendArp_start_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 14]: Pending (id: calypso-VE_start_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 3 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 15]: Pending (id: epione-FS_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: Pending (id: essex02-LVM_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 4 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 17]: Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 10]: Pending (id: calypso-FS_stop_0, loc: hazel, priority: 0)
Below: we want to start essex02-LVM on fig,
so we need to stop it on hazel first.
Above: to stop essex02-LVM, need to stop calypso-FS first.
We cannot, as calypso-FS is unmanaged.
Too bad.
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 5 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 18]: Pending (id: essex02-LVM_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 27]: Pending (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 46]: Pending (id: ms-drbd-essex02_promoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 50]: Pending (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 6 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 21]: Pending (id: essextest-FS_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: Pending (id: essex02-LVM_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 7 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 22]: Pending (id: essextest-SendArp_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 23]: Pending (id: essextest-VE_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 8 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 23]: Pending (id: essextest-VE_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 21]: Pending (id: essextest-FS_start_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 9 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 27]: Pending (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 45]: Pending (id: ms-drbd-essex02_promote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 10 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 28]: Pending (id: essex02-DRBD:0_monitor_29000, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 27]: Pending (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 50]: Pending (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 56]: Pending (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 11 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 115]: Pending (id: essex02-DRBD:0_pre_notify_promote_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 47]: Pending (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 12 is pending (priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 116]: Pending (id: essex02-DRBD:0_post_notify_promote_0, loc: fig, priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 49]: Pending (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 13 was confirmed (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 14 is pending (priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 120]: Pending (id: essex02-DRBD:0_post_notify_demote_0, loc: fig, priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 55]: Pending (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 15 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 31]: Pending (id: essex02-DRBD:1_demote_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 51]: Pending (id: ms-drbd-essex02_demote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 16 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 32]: Pending (id: essex02-DRBD:1_monitor_31000, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 31]: Pending (id: essex02-DRBD:1_demote_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 50]: Pending (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 56]: Pending (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 17 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 117]: Pending (id: essex02-DRBD:1_pre_notify_promote_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 47]: Pending (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 18 is pending (priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 118]: Pending (id: essex02-DRBD:1_post_notify_promote_0, loc: hazel, priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 49]: Pending (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 19 was confirmed (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 20 is pending (priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 122]: Pending (id: essex02-DRBD:1_post_notify_demote_0, loc: hazel, priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 55]: Pending (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 21 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 45]: Pending (id: ms-drbd-essex02_promote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 48]: Pending (id: ms-drbd-essex02_confirmed-pre_notify_promote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 52]: Pending (id: ms-drbd-essex02_demoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 22 is pending (priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 46]: Pending (id: ms-drbd-essex02_promoted_0, type: pseduo, priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 27]: Pending (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 23 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 47]: Pending (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 56]: Pending (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 24 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 48]: Pending (id: ms-drbd-essex02_confirmed-pre_notify_promote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 47]: Pending (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 115]: Pending (id: essex02-DRBD:0_pre_notify_promote_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 117]: Pending (id: essex02-DRBD:1_pre_notify_promote_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 25 is pending (priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 49]: Pending (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 46]: Pending (id: ms-drbd-essex02_promoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 48]: Pending (id: ms-drbd-essex02_confirmed-pre_notify_promote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 26 is pending (priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 50]: Pending (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 49]: Pending (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 116]: Pending (id: essex02-DRBD:0_post_notify_promote_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 118]: Pending (id: essex02-DRBD:1_post_notify_promote_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 27 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 51]: Pending (id: ms-drbd-essex02_demote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 54]: Completed (id: ms-drbd-essex02_confirmed-pre_notify_demote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 28 is pending (priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 52]: Pending (id: ms-drbd-essex02_demoted_0, type: pseduo, priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 31]: Pending (id: essex02-DRBD:1_demote_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 51]: Pending (id: ms-drbd-essex02_demote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 29 was confirmed (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 30 was confirmed (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 31 is pending (priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 55]: Pending (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 52]: Pending (id: ms-drbd-essex02_demoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 54]: Completed (id: ms-drbd-essex02_confirmed-pre_notify_demote_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 32 is pending (priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 56]: Pending (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, priority: 1000000)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 55]: Pending (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 120]: Pending (id: essex02-DRBD:0_post_notify_demote_0, loc: fig, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 122]: Pending (id: essex02-DRBD:1_post_notify_demote_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 33 is pending (priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 2]: Pending (id: all_stopped, type: pseduo, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0)
> Apr 27 11:06:36 fig crmd: [395]: info: te_graph_trigger: Transition 121 is now complete
> Apr 27 11:06:36 fig crmd: [395]: info: notify_crmd: Transition 121 status: done - <null>
> Apr 27 11:06:36 fig crmd: [395]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
> Apr 27 11:06:36 fig crmd: [395]: info: do_state_transition: Starting PEngine Recheck Timer
>
>
> The configuration is as follows:
>
>
> node fig
> node hazel
> primitive artemis-FS ocf:heartbeat:Filesystem \
> params device="/dev/essex02/artemis"
> directory="/containers/artemis" fstype="ext4" options="relatime,discard" \
> op start interval="0" timeout="60s" \
> op start interval="0" timeout="60s"
> primitive artemis-SendArp ocf:heartbeat:SendArp \
> params ip="XYZ.XY.126.102" nic="eth0"
> primitive artemis-VE ocf:heartbeat:ManageVE \
> params veid="2501" \
> op start interval="0" timeout="75s" \
> op start interval="0" timeout="75s" \
> meta target-role="Stopped" is-managed="false"
> primitive athena-FS ocf:heartbeat:Filesystem \
> params device="/dev/essex02/athena"
> directory="/containers/athena" fstype="ext4" options="relatime,discard" \
> op start interval="0" timeout="60s" \
> op start interval="0" timeout="60s" \
> meta is-managed="true" target-role="Started"
> primitive athena-SendArp ocf:heartbeat:SendArp \
> params ip="XYZ.XY.118.23" nic="eth0"
> primitive athena-VE ocf:heartbeat:ManageVE \
> params veid="2504" \
> op start interval="0" timeout="75s" \
> op start interval="0" timeout="75s" \
> meta target-role="Stopped" is-managed="false"
> primitive calypso-FS ocf:heartbeat:Filesystem \
> params device="/dev/essex03/calypso"
> directory="/containers/calypso" fstype="ext4" options="relatime,discard" \
> op start interval="0" timeout="60s" \
> op start interval="0" timeout="60s" \
> meta target-role="Started" is-managed="false"
> primitive calypso-SendArp ocf:heartbeat:SendArp \
> params ip="XYZ.XY.126.110" nic="eth0"
> primitive calypso-VE ocf:heartbeat:ManageVE \
> params veid="2000" \
> op start interval="0" timeout="75s" \
> op start interval="0" timeout="75s" \
> meta target-role="Stopped" is-managed="false"
> primitive epione-FS ocf:heartbeat:Filesystem \
> params device="/dev/essex02/epione"
> directory="/containers/epione" fstype="ext4" options="relatime,discard" \
> op start interval="0" timeout="60s" \
> op start interval="0" timeout="60s" \
> meta target-role="Started"
> primitive epione-SendArp ocf:heartbeat:SendArp \
> params ip="XYZ.XY.118.21" nic="eth0"
> primitive epione-VE ocf:heartbeat:ManageVE \
> params veid="2502" \
> op start interval="0" timeout="75s" \
> op start interval="0" timeout="75s" \
> meta target-role="Stopped" is-managed="false"
> primitive essex02-DRBD ocf:linbit:drbd \
> params drbd_resource="essex02" \
> op monitor interval="29s" role="Master" \
> op monitor interval="31s" role="Slave" \
> op start interval="0" timeout="240s" \
> op start interval="0" timeout="100s"
> primitive essex02-LVM ocf:heartbeat:LVM \
> params volgrpname="essex02" \
> op start interval="0" timeout="240s" \
> op start interval="0" timeout="100s" \
> meta target-role="Started" is-managed="true"
> primitive essex03-DRBD ocf:linbit:drbd \
> params drbd_resource="essex03" \
> op monitor interval="29s" role="Master" \
> op monitor interval="31s" role="Slave" \
> op start interval="0" timeout="240s" \
> meta target-role="Started" is-managed="false"
> primitive essex03-LVM ocf:heartbeat:LVM \
> params volgrpname="essex03" \
> op start interval="0" timeout="240s" \
> op start interval="0" timeout="100s" \
> meta target-role="Started" is-managed="false"
> primitive essextest-FS ocf:heartbeat:Filesystem \
> params device="/dev/essex02/essextest"
> directory="/containers/essextest" fstype="ext4" options="relatime,discard" \
> op start interval="0" timeout="60s" \
> op start interval="0" timeout="60s" \
> meta target-role="Started"
> primitive essextest-SendArp ocf:heartbeat:SendArp \
> params ip="XYZ.XY.118.25" nic="eth0" \
> meta target-role="Started"
> primitive essextest-VE ocf:heartbeat:ManageVE \
> params veid="2599" \
> op start interval="0" timeout="75s" \
> op start interval="0" timeout="75s" \
> meta target-role="Started"
> ms ms-drbd-essex02 essex02-DRBD \
> meta master-max="1" notify="true" globally-unique="false"
> target-role="Started" is-managed="true"
> ms ms-drbd-essex03 essex03-DRBD \
> meta master-max="1" master-node-max="1" clone-max="2"
> clone-node-max="1" notify="true" globally-unique="false"
> target-role="Started" is-managed="false"
> location ms-drbd-essex02-on-fig ms-drbd-essex02 \
> rule $id="ms-drbd-essex02-on-fig-rule" $role="master" 100:
> #uname eq fig
> location ms-drbd-essex03-on-hazel ms-drbd-essex03 \
> rule $id="ms-drbd-essex03-on-hazel-rule" $role="master" 100:
> #uname eq hazel
> colocation artemis-FS-with-essex02-LVM inf: artemis-FS essex02-LVM
> colocation artemis-SendArp-with-artemis-VE inf: artemis-SendArp artemis-VE
> colocation artemis-VE-with-artemis-FS inf: artemis-VE artemis-FS
> colocation athena-FS-with-essex02-LVM inf: athena-FS essex02-LVM
> colocation athena-SendArp-with-athena-VE inf: athena-SendArp athena-VE
> colocation athena-VE-with-athena-FS inf: athena-VE athena-FS
> colocation calypso-FS-on-essex03-LVM inf: calypso-FS essex03-LVM
Ok, colo calypso with essex03... but then, why ...
> colocation calypso-SendArp-with-calypso-VE inf: calypso-SendArp calypso-VE
> colocation calypso-VE-with-calypso-FS inf: calypso-VE calypso-FS
> colocation epione-FS-on-essex02-LVM inf: epione-FS essex02-LVM
> colocation epione-FS-with-essex02-LVM inf: epione-FS essex02-LVM
> colocation epione-SendArp-with-epione-VE inf: epione-SendArp epione-VE
> colocation epione-VE-with-epione-FS inf: epione-VE epione-FS
> colocation essex02-LVM-with-essex02-DRBD-Master inf: essex02-LVM
> ms-drbd-essex02:Master
> colocation essex03LVM-on-ms-drbd-essex03 inf: essex03-LVM
> ms-drbd-essex03:Master
> colocation essextest-FS-with-essex02-LVM inf: essextest-FS essex02-LVM
> colocation essextest-SendArp-with-essextest-VE inf: essextest-SendArp
> essextest-VE
> colocation essextest-VE-with-essextest-FS inf: essextest-VE essextest-FS
> order artemis-FS-before-artemis-VE inf: artemis-FS artemis-VE
> order artemis-VE-before-artemis-SendArp inf: artemis-VE artemis-SendArp
> order athena-FS-before-athena-VE inf: athena-FS athena-VE
> order athena-VE-before-athena-SendArp inf: athena-VE athena-SendArp
> order calypso-FS-before-calypso-VE inf: calypso-FS calypso-VE
> order calypso-VE-before-calypso-SendArp inf: calypso-VE calypso-SendArp
> order epione-FS-before-epione-VE inf: epione-FS epione-VE
> order epione-VE-before-epione-SendArp inf: epione-VE epione-SendArp
> order essex02-lvm-before-artemis-FS inf: essex02-LVM artemis-FS
> order essex02-lvm-before-athena-FS inf: essex02-LVM athena-FS
> order essex02-lvm-before-calypso-FS inf: essex02-LVM calypso-FS
Order essex02 with calypso? typo? is this supposed to be essex03?
> order essex02-lvm-before-epione-FS inf: essex02-LVM epione-FS
> order essex02-lvm-before-essextest-FS inf: essex02-LVM essextest-FS
> order essextest-FS-before-essextest-VE inf: essextest-FS essextest-VE
> order essextest-VE-before-essextest-SendArp inf: essextest-VE
> essextest-SendArp
> order ms-drbd-essex02-before-lvm inf: ms-drbd-essex02:promote
> essex02-LVM:start
> order ms-drbd-essex03-before-lvm inf: ms-drbd-essex03:promote
> essex03-LVM:start
> property $id="cib-bootstrap-options" \
> dc-version="1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b" \
> cluster-infrastructure="openais" \
> expected-quorum-votes="2" \
> no-quorum-policy="ignore" \
> stonith-enabled="false" \
> last-lrm-refresh="1335487560"
>
>
> # crm configure verify
> WARNING: artemis-FS: default timeout 20s for stop is smaller than the
> advised 60
> WARNING: artemis-VE: default timeout 20s for stop is smaller than the
> advised 75
> WARNING: athena-FS: default timeout 20s for stop is smaller than the
> advised 60
> WARNING: athena-VE: default timeout 20s for stop is smaller than the
> advised 75
> WARNING: calypso-FS: default timeout 20s for stop is smaller than the
> advised 60
> WARNING: calypso-VE: default timeout 20s for stop is smaller than the
> advised 75
> WARNING: epione-FS: default timeout 20s for stop is smaller than the
> advised 60
> WARNING: epione-VE: default timeout 20s for stop is smaller than the
> advised 75
> WARNING: essex02-LVM: default timeout 20s for stop is smaller than the
> advised 30
> WARNING: essex03-LVM: default timeout 20s for stop is smaller than the
> advised 30
> WARNING: essextest-FS: default timeout 20s for stop is smaller than the
> advised 60
> WARNING: essextest-VE: default timeout 20s for stop is smaller than the
> advised 75
> WARNING: essex02-DRBD: specified timeout 100s for start is smaller than
> the advised 240
> WARNING: essex02-DRBD: default timeout 20s for stop is smaller than the
> advised 100
> WARNING: essex03-DRBD: default timeout 20s for stop is smaller than the
> advised 100
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
More information about the Pacemaker
mailing list