May 28 11:00:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:39 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:00:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:00:54 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:00:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:00:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:10 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:01:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:01:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:01:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:02:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:02:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:02:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:02:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:02:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 78
May 28 11:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797425057s to complete
May 28 11:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797425057s to complete
May 28 11:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797425056s to complete
May 28 11:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797425056s to complete
May 28 11:02:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797425056s to complete
May 28 11:02:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797425056s to complete
May 28 11:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797425056s to complete
May 28 11:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797425056s to complete
May 28 11:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797425056s to complete
May 28 11:02:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 26587 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:02:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-79: Waiting on 1 outstanding join acks
May 28 11:02:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243519343-15 
May 28 11:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797425055s to complete
May 28 11:02:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:02:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 26588 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:02:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243519343 
May 28 11:02:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:02:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:40 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:02:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:02:55 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:02:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:02:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:03:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:03:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:25 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:04:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:40 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:04:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:04:55 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:04:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:04:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:04 node2 crmd: [10635]: info: crm_timer_popped: PEngine Recheck Timer (I_PE_CALC) just popped!
May 28 11:05:04 node2 crmd: [10635]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:05:04 node2 crmd: [10635]: WARN: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED
May 28 11:05:04 node2 crmd: [10635]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797409104s to complete
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797409104s to complete
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797409104s to complete
May 28 11:05:04 node2 crmd: [10635]: info: do_pe_invoke: Query 103: Requesting the current CIB: S_POLICY_ENGINE
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_PE_INVOKE took 797409104s to complete
May 28 11:05:04 node2 crmd: [10635]: info: do_pe_invoke_callback: Invoking the PE: ref=pe_calc-dc-1243519504-64, seq=2, quorate=1
May 28 11:05:04 node2 pengine: [10647]: WARN: unpack_resources: No STONITH resources have been defined
May 28 11:05:04 node2 pengine: [10647]: info: determine_online_status: Node node2 is online
May 28 11:05:04 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node2 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:05:04 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node2
May 28 11:05:04 node2 pengine: [10647]: info: determine_online_status: Node node1 is online
May 28 11:05:04 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node1 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:05:04 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node1
May 28 11:05:04 node2 pengine: [10647]: notice: native_print: failover-ip	(ocf::heartbeat:IPaddr):	Started node1
May 28 11:05:04 node2 pengine: [10647]: notice: native_print: scriptHa	(lsb:scriptHa.sh):	Started node2
May 28 11:05:04 node2 pengine: [10647]: info: get_failcount: scriptHa has failed 2 times on node2
May 28 11:05:04 node2 pengine: [10647]: notice: common_apply_stickiness: scriptHa can fail 999998 more times on node2 before being forced off
May 28 11:05:04 node2 pengine: [10647]: notice: LogActions: Leave resource failover-ip	(Started node1)
May 28 11:05:04 node2 pengine: [10647]: notice: LogActions: Leave resource scriptHa	(Started node2)
May 28 11:05:04 node2 pengine: [10647]: info: process_pe_message: Transition 25: PEngine Input stored in: /var/lib/pengine/pe-input-530.bz2
May 28 11:05:04 node2 pengine: [10647]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797409102s to complete
May 28 11:05:04 node2 crmd: [10635]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797409102s to complete
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797409102s to complete
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797409102s to complete
May 28 11:05:04 node2 crmd: [10635]: info: unpack_graph: Unpacked transition 25: 0 actions in 0 synapses
May 28 11:05:04 node2 crmd: [10635]: info: do_te_invoke: Processing graph 25 (ref=pe_calc-dc-1243519504-64) derived from /var/lib/pengine/pe-input-530.bz2
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_TE_INVOKE took 797409102s to complete
May 28 11:05:04 node2 crmd: [10635]: info: run_graph: ====================================================
May 28 11:05:04 node2 crmd: [10635]: notice: run_graph: Transition 25 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-530.bz2): Complete
May 28 11:05:04 node2 crmd: [10635]: info: te_graph_trigger: Transition 25 is now complete
May 28 11:05:04 node2 crmd: [10635]: info: notify_crmd: Transition 25 status: done - <null>
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797409101s to complete
May 28 11:05:04 node2 crmd: [10635]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
May 28 11:05:04 node2 crmd: [10635]: info: do_state_transition: Starting PEngine Recheck Timer
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797409101s to complete
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797409101s to complete
May 28 11:05:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797409101s to complete
May 28 11:05:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:05:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:05:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:05:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:05:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:05:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 79
May 28 11:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797407242s to complete
May 28 11:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797407242s to complete
May 28 11:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797407242s to complete
May 28 11:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797407242s to complete
May 28 11:05:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797407242s to complete
May 28 11:05:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797407242s to complete
May 28 11:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797407242s to complete
May 28 11:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797407242s to complete
May 28 11:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797407242s to complete
May 28 11:05:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 26725 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:05:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-80: Waiting on 1 outstanding join acks
May 28 11:05:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243519523-16 
May 28 11:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797407241s to complete
May 28 11:05:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:05:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 26726 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:05:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243519523 
May 28 11:05:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:05:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:05:56 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:05:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:05:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:11 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:06:11 node2 cib: [10631]: info: cib_stats: Processed 1 operations (0.00us average, 0% utilization) in the last 10min
May 28 11:06:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:26 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:06:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:41 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:06:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:06:56 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:06:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:06:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:07:56 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:07:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:07:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:11 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:08:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:08:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:08:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:08:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:08:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:08:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 80
May 28 11:08:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797389427s to complete
May 28 11:08:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797389427s to complete
May 28 11:08:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797389427s to complete
May 28 11:08:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797389427s to complete
May 28 11:08:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:08:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797389427s to complete
May 28 11:08:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:08:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797389427s to complete
May 28 11:08:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797389427s to complete
May 28 11:08:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797389427s to complete
May 28 11:08:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797389427s to complete
May 28 11:08:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 26865 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:08:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-81: Waiting on 1 outstanding join acks
May 28 11:08:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243519703-16 
May 28 11:08:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797389426s to complete
May 28 11:08:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:08:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 26866 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:08:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243519703 
May 28 11:08:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:08:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:26 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:08:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:08:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:08:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:41 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:09:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:09:57 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:09:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:09:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:12 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:10:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:27 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:10:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:10:57 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:10:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:10:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:12 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:11:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:11:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:11:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:11:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:11:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:11:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 81
May 28 11:11:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797371612s to complete
May 28 11:11:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797371612s to complete
May 28 11:11:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797371612s to complete
May 28 11:11:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797371612s to complete
May 28 11:11:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:11:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797371612s to complete
May 28 11:11:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:11:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797371612s to complete
May 28 11:11:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797371612s to complete
May 28 11:11:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797371612s to complete
May 28 11:11:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797371612s to complete
May 28 11:11:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27003 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:11:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-82: Waiting on 1 outstanding join acks
May 28 11:11:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243519883-16 
May 28 11:11:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797371611s to complete
May 28 11:11:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:11:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27004 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:11:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243519883 
May 28 11:11:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:11:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:27 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:11:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:42 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:11:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:11:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:11:57 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:12:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:12 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:12:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:12:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:12:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:27 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:13:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:42 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:13:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:13:57 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:13:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:13:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:14:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:14:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:14:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:14:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:14:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 82
May 28 11:14:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797353798s to complete
May 28 11:14:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797353798s to complete
May 28 11:14:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797353798s to complete
May 28 11:14:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797353798s to complete
May 28 11:14:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:14:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797353798s to complete
May 28 11:14:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:14:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797353798s to complete
May 28 11:14:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797353798s to complete
May 28 11:14:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797353798s to complete
May 28 11:14:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797353798s to complete
May 28 11:14:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27141 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:14:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-83: Waiting on 1 outstanding join acks
May 28 11:14:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243520063-16 
May 28 11:14:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797353797s to complete
May 28 11:14:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:14:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27142 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:14:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243520063 
May 28 11:14:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:14:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:43 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:14:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:14:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:14:58 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:15:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:13 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:15:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:28 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:15:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:43 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:15:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:15:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:15:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:43 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:16:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:16:58 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:16:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:16:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:13 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:17:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:17:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:17:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:17:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:17:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:17:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 83
May 28 11:17:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797335983s to complete
May 28 11:17:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797335983s to complete
May 28 11:17:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797335983s to complete
May 28 11:17:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797335983s to complete
May 28 11:17:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:17:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797335983s to complete
May 28 11:17:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:17:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797335983s to complete
May 28 11:17:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797335983s to complete
May 28 11:17:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797335983s to complete
May 28 11:17:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797335983s to complete
May 28 11:17:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27279 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:17:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-84: Waiting on 1 outstanding join acks
May 28 11:17:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243520243-16 
May 28 11:17:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797335982s to complete
May 28 11:17:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:17:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27280 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:17:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243520243 
May 28 11:17:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:17:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:28 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:17:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:17:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:17:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:29 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:18:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:44 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:18:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:18:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:18:59 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:19:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:14 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:19:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:29 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:19:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:19:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:19:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:04 node2 crmd: [10635]: info: crm_timer_popped: PEngine Recheck Timer (I_PE_CALC) just popped!
May 28 11:20:04 node2 crmd: [10635]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:20:04 node2 crmd: [10635]: WARN: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED
May 28 11:20:04 node2 crmd: [10635]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797320029s to complete
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797320029s to complete
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797320029s to complete
May 28 11:20:04 node2 crmd: [10635]: info: do_pe_invoke: Query 104: Requesting the current CIB: S_POLICY_ENGINE
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_PE_INVOKE took 797320029s to complete
May 28 11:20:04 node2 crmd: [10635]: info: do_pe_invoke_callback: Invoking the PE: ref=pe_calc-dc-1243520404-65, seq=2, quorate=1
May 28 11:20:04 node2 pengine: [10647]: WARN: unpack_resources: No STONITH resources have been defined
May 28 11:20:04 node2 pengine: [10647]: info: determine_online_status: Node node2 is online
May 28 11:20:04 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node2 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:20:04 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node2
May 28 11:20:04 node2 pengine: [10647]: info: determine_online_status: Node node1 is online
May 28 11:20:04 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node1 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:20:04 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node1
May 28 11:20:04 node2 pengine: [10647]: notice: native_print: failover-ip	(ocf::heartbeat:IPaddr):	Started node1
May 28 11:20:04 node2 pengine: [10647]: notice: native_print: scriptHa	(lsb:scriptHa.sh):	Started node2
May 28 11:20:04 node2 pengine: [10647]: info: get_failcount: scriptHa has failed 2 times on node2
May 28 11:20:04 node2 pengine: [10647]: notice: common_apply_stickiness: scriptHa can fail 999998 more times on node2 before being forced off
May 28 11:20:04 node2 pengine: [10647]: notice: LogActions: Leave resource failover-ip	(Started node1)
May 28 11:20:04 node2 pengine: [10647]: notice: LogActions: Leave resource scriptHa	(Started node2)
May 28 11:20:04 node2 pengine: [10647]: info: process_pe_message: Transition 26: PEngine Input stored in: /var/lib/pengine/pe-input-531.bz2
May 28 11:20:04 node2 pengine: [10647]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797320027s to complete
May 28 11:20:04 node2 crmd: [10635]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797320027s to complete
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797320027s to complete
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797320027s to complete
May 28 11:20:04 node2 crmd: [10635]: info: unpack_graph: Unpacked transition 26: 0 actions in 0 synapses
May 28 11:20:04 node2 crmd: [10635]: info: do_te_invoke: Processing graph 26 (ref=pe_calc-dc-1243520404-65) derived from /var/lib/pengine/pe-input-531.bz2
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_TE_INVOKE took 797320027s to complete
May 28 11:20:04 node2 crmd: [10635]: info: run_graph: ====================================================
May 28 11:20:04 node2 crmd: [10635]: notice: run_graph: Transition 26 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-531.bz2): Complete
May 28 11:20:04 node2 crmd: [10635]: info: te_graph_trigger: Transition 26 is now complete
May 28 11:20:04 node2 crmd: [10635]: info: notify_crmd: Transition 26 status: done - <null>
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797320026s to complete
May 28 11:20:04 node2 crmd: [10635]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
May 28 11:20:04 node2 crmd: [10635]: info: do_state_transition: Starting PEngine Recheck Timer
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797320026s to complete
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797320026s to complete
May 28 11:20:04 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797320026s to complete
May 28 11:20:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:20:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:20:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:20:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:20:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:20:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 84
May 28 11:20:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797318169s to complete
May 28 11:20:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797318169s to complete
May 28 11:20:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797318169s to complete
May 28 11:20:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797318169s to complete
May 28 11:20:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:20:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797318169s to complete
May 28 11:20:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:20:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797318168s to complete
May 28 11:20:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797318168s to complete
May 28 11:20:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797318168s to complete
May 28 11:20:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797318168s to complete
May 28 11:20:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27417 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:20:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-85: Waiting on 1 outstanding join acks
May 28 11:20:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243520423-17 
May 28 11:20:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797318167s to complete
May 28 11:20:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:20:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27418 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:20:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243520423 
May 28 11:20:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:20:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:29 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:20:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:44 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:20:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:20:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:20:59 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:21:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:14 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:21:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:21:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:21:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:29 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:22:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:44 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:22:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:22:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:22:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:00 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:23:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:15 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:23:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:23:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:23:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:23:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:23:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:23:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 85
May 28 11:23:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797300354s to complete
May 28 11:23:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797300354s to complete
May 28 11:23:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797300354s to complete
May 28 11:23:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797300354s to complete
May 28 11:23:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:23:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797300354s to complete
May 28 11:23:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:23:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797300354s to complete
May 28 11:23:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797300354s to complete
May 28 11:23:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797300354s to complete
May 28 11:23:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797300354s to complete
May 28 11:23:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27557 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:23:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-86: Waiting on 1 outstanding join acks
May 28 11:23:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243520603-17 
May 28 11:23:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797300353s to complete
May 28 11:23:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:23:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27558 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:23:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243520603 
May 28 11:23:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:23:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:23:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:23:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:00 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:24:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:15 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:24:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:30 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:24:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:45 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:24:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:24:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:24:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:00 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:25:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:25:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:25:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:00 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:26:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:11 node2 cib: [10631]: info: cib_stats: Processed 1 operations (0.00us average, 0% utilization) in the last 10min
May 28 11:26:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:15 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:26:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:26:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:26:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:26:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:26:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:26:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 86
May 28 11:26:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797282539s to complete
May 28 11:26:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797282539s to complete
May 28 11:26:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797282539s to complete
May 28 11:26:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797282539s to complete
May 28 11:26:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:26:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797282539s to complete
May 28 11:26:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:26:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797282539s to complete
May 28 11:26:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797282539s to complete
May 28 11:26:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797282539s to complete
May 28 11:26:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797282539s to complete
May 28 11:26:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27695 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:26:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-87: Waiting on 1 outstanding join acks
May 28 11:26:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243520783-17 
May 28 11:26:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797282538s to complete
May 28 11:26:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:26:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27696 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:26:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243520783 
May 28 11:26:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:26:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:30 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:26:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:26:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:26:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:46 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:27:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:27:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:27:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:01 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:28:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:16 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:28:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:31 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:28:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:28:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:28:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:29:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:29:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:29:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:29:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:29:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 87
May 28 11:29:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797264725s to complete
May 28 11:29:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797264725s to complete
May 28 11:29:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797264725s to complete
May 28 11:29:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797264725s to complete
May 28 11:29:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:29:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797264725s to complete
May 28 11:29:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:29:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797264724s to complete
May 28 11:29:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797264724s to complete
May 28 11:29:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797264724s to complete
May 28 11:29:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797264724s to complete
May 28 11:29:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27833 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:29:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-88: Waiting on 1 outstanding join acks
May 28 11:29:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243520963-17 
May 28 11:29:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797264723s to complete
May 28 11:29:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:29:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27834 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:29:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243520963 
May 28 11:29:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:29:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:31 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 21852: binary operator expected 
May 28 11:29:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: - <cib admin_epoch="0" epoch="117" num_updates="9" >
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: -   <configuration >
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: -     <resources >
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: -       <primitive id="scriptHa" >
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: -         <meta_attributes id="scriptHa-meta_attributes" >
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: -           <nvpair value="Started" id="scriptHa-meta_attributes-target-role" />
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: -         </meta_attributes>
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: -       </primitive>
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: -     </resources>
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: -   </configuration>
May 28 11:29:35 node2 crmd: [10635]: info: abort_transition_graph: need_abort:59 - Triggered transition abort (complete=1) : Non-status change
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: - </cib>
May 28 11:29:35 node2 crmd: [10635]: info: need_abort: Aborting on change to admin_epoch
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: + <cib admin_epoch="0" epoch="118" num_updates="1" >
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: +   <configuration >
May 28 11:29:35 node2 crmd: [10635]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: +     <resources >
May 28 11:29:35 node2 crmd: [10635]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: +       <primitive id="scriptHa" >
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: +         <meta_attributes id="scriptHa-meta_attributes" >
May 28 11:29:35 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797263571s to complete
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: +           <nvpair value="Stopped" id="scriptHa-meta_attributes-target-role" />
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: +         </meta_attributes>
May 28 11:29:35 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797263571s to complete
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: +       </primitive>
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: +     </resources>
May 28 11:29:35 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797263571s to complete
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: +   </configuration>
May 28 11:29:35 node2 cib: [10631]: info: log_data_element: cib:diff: + </cib>
May 28 11:29:35 node2 crmd: [10635]: info: do_pe_invoke: Query 105: Requesting the current CIB: S_POLICY_ENGINE
May 28 11:29:35 node2 cib: [10631]: info: cib_process_request: Operation complete: op cib_modify for section resources (origin=node1/crm_resource/4, version=0.118.1): ok (rc=0)
May 28 11:29:35 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_PE_INVOKE took 797263571s to complete
May 28 11:29:35 node2 crmd: [10635]: info: do_pe_invoke_callback: Invoking the PE: ref=pe_calc-dc-1243520975-66, seq=2, quorate=1
May 28 11:29:35 node2 cib: [27846]: info: write_cib_contents: Archived previous version as /var/lib/heartbeat/crm/cib-36.raw
May 28 11:29:35 node2 pengine: [10647]: WARN: unpack_resources: No STONITH resources have been defined
May 28 11:29:35 node2 pengine: [10647]: info: determine_online_status: Node node2 is online
May 28 11:29:35 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node2 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:29:35 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node2
May 28 11:29:35 node2 pengine: [10647]: info: determine_online_status: Node node1 is online
May 28 11:29:35 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node1 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:29:35 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node1
May 28 11:29:35 node2 pengine: [10647]: notice: native_print: failover-ip	(ocf::heartbeat:IPaddr):	Started node1
May 28 11:29:35 node2 pengine: [10647]: notice: native_print: scriptHa	(lsb:scriptHa.sh):	Started node2
May 28 11:29:35 node2 pengine: [10647]: info: get_failcount: scriptHa has failed 2 times on node2
May 28 11:29:35 node2 pengine: [10647]: notice: common_apply_stickiness: scriptHa can fail 999998 more times on node2 before being forced off
May 28 11:29:35 node2 cib: [27846]: info: write_cib_contents: Wrote version 0.118.0 of the CIB to disk (digest: 1043166aa10ed9d54e6066d6694a399a)
May 28 11:29:35 node2 cib: [27846]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.BM3IZT (digest: /var/lib/heartbeat/crm/cib.fBH4Ps)
May 28 11:29:35 node2 pengine: [10647]: WARN: native_color: Resource scriptHa cannot run anywhere
May 28 11:29:35 node2 pengine: [10647]: notice: LogActions: Leave resource failover-ip	(Started node1)
May 28 11:29:35 node2 pengine: [10647]: notice: LogActions: Stop resource scriptHa	(node2)
May 28 11:29:35 node2 cib: [10631]: info: Managed write_cib_contents process 27846 exited with return code 0.
May 28 11:29:35 node2 pengine: [10647]: WARN: process_pe_message: Transition 27: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/pengine/pe-warn-234.bz2
May 28 11:29:35 node2 pengine: [10647]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
May 28 11:29:35 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797263566s to complete
May 28 11:29:35 node2 crmd: [10635]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
May 28 11:29:35 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797263566s to complete
May 28 11:29:35 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797263566s to complete
May 28 11:29:35 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797263566s to complete
May 28 11:29:35 node2 crmd: [10635]: info: unpack_graph: Unpacked transition 27: 2 actions in 2 synapses
May 28 11:29:35 node2 crmd: [10635]: info: do_te_invoke: Processing graph 27 (ref=pe_calc-dc-1243520975-66) derived from /var/lib/pengine/pe-warn-234.bz2
May 28 11:29:35 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_TE_INVOKE took 797263566s to complete
May 28 11:29:35 node2 crmd: [10635]: info: te_rsc_command: Initiating action 9: stop scriptHa_stop_0 on node2 (local)
May 28 11:29:35 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=9:27:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_stop_0 )
May 28 11:29:35 node2 lrmd: [10638]: info: rsc:scriptHa: stop
May 28 11:29:35 node2 lrmd: [27847]: WARN: For LSB init script, no additional parameters are needed.
May 28 11:29:35 node2 lrmd: [10638]: info: RA output: (scriptHa:stop:stdout) Stopping RSC-TEST...
May 28 11:29:35 node2 lrmd: [10638]: info: RA output: (scriptHa:stop:stderr) /etc/init.d/scriptHa.sh: line 14: [: 21852: binary operator expected /etc/init.d/scriptHa.sh: line 18: kill: (27850) - No such process 
May 28 11:29:35 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_monitor_15000 (call=18, rc=-2, cib-update=0, confirmed=true) Cancelled unknown exec error
May 28 11:29:36 node2 lrmd: [10638]: info: RA output: (scriptHa:stop:stdout) .. Done. 
May 28 11:29:36 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_stop_0 (call=19, rc=0, cib-update=106, confirmed=true) complete ok
May 28 11:29:36 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_stop_0 (9) confirmed on node2 (rc=0)
May 28 11:29:36 node2 crmd: [10635]: info: te_pseudo_action: Pseudo action 3 fired and confirmed
May 28 11:29:36 node2 crmd: [10635]: info: run_graph: ====================================================
May 28 11:29:36 node2 crmd: [10635]: notice: run_graph: Transition 27 (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-warn-234.bz2): Complete
May 28 11:29:36 node2 crmd: [10635]: info: te_graph_trigger: Transition 27 is now complete
May 28 11:29:36 node2 crmd: [10635]: info: notify_crmd: Transition 27 status: done - <null>
May 28 11:29:36 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797263460s to complete
May 28 11:29:36 node2 crmd: [10635]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
May 28 11:29:36 node2 crmd: [10635]: info: do_state_transition: Starting PEngine Recheck Timer
May 28 11:29:36 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797263460s to complete
May 28 11:29:36 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797263460s to complete
May 28 11:29:36 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797263460s to complete
May 28 11:29:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: - <cib admin_epoch="0" epoch="118" num_updates="2" >
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: -   <configuration >
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: -     <resources >
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: -       <primitive id="scriptHa" >
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: -         <meta_attributes id="scriptHa-meta_attributes" >
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: -           <nvpair value="Stopped" id="scriptHa-meta_attributes-target-role" />
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: -         </meta_attributes>
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: -       </primitive>
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: -     </resources>
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: -   </configuration>
May 28 11:29:46 node2 crmd: [10635]: info: abort_transition_graph: need_abort:59 - Triggered transition abort (complete=1) : Non-status change
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: - </cib>
May 28 11:29:46 node2 crmd: [10635]: info: need_abort: Aborting on change to admin_epoch
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: + <cib admin_epoch="0" epoch="119" num_updates="1" >
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: +   <configuration >
May 28 11:29:46 node2 crmd: [10635]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: +     <resources >
May 28 11:29:46 node2 crmd: [10635]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: +       <primitive id="scriptHa" >
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: +         <meta_attributes id="scriptHa-meta_attributes" >
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797262481s to complete
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: +           <nvpair value="Started" id="scriptHa-meta_attributes-target-role" />
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: +         </meta_attributes>
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797262481s to complete
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: +       </primitive>
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: +     </resources>
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797262481s to complete
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: +   </configuration>
May 28 11:29:46 node2 cib: [10631]: info: log_data_element: cib:diff: + </cib>
May 28 11:29:46 node2 crmd: [10635]: info: do_pe_invoke: Query 107: Requesting the current CIB: S_POLICY_ENGINE
May 28 11:29:46 node2 cib: [10631]: info: cib_process_request: Operation complete: op cib_modify for section resources (origin=node1/crm_resource/4, version=0.119.1): ok (rc=0)
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_PE_INVOKE took 797262481s to complete
May 28 11:29:46 node2 crmd: [10635]: info: do_pe_invoke_callback: Invoking the PE: ref=pe_calc-dc-1243520986-68, seq=2, quorate=1
May 28 11:29:46 node2 cib: [27853]: info: write_cib_contents: Archived previous version as /var/lib/heartbeat/crm/cib-37.raw
May 28 11:29:46 node2 cib: [27853]: info: write_cib_contents: Wrote version 0.119.0 of the CIB to disk (digest: 98449476b712cf0cb08e26b581e49408)
May 28 11:29:46 node2 pengine: [10647]: WARN: unpack_resources: No STONITH resources have been defined
May 28 11:29:46 node2 pengine: [10647]: info: determine_online_status: Node node2 is online
May 28 11:29:46 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node2 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:29:46 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node2
May 28 11:29:46 node2 pengine: [10647]: info: determine_online_status: Node node1 is online
May 28 11:29:46 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node1 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:29:46 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node1
May 28 11:29:46 node2 pengine: [10647]: notice: native_print: failover-ip	(ocf::heartbeat:IPaddr):	Started node1
May 28 11:29:46 node2 pengine: [10647]: notice: native_print: scriptHa	(lsb:scriptHa.sh):	Stopped 
May 28 11:29:46 node2 pengine: [10647]: info: get_failcount: scriptHa has failed 2 times on node2
May 28 11:29:46 node2 pengine: [10647]: notice: common_apply_stickiness: scriptHa can fail 999998 more times on node2 before being forced off
May 28 11:29:46 node2 cib: [27853]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.Gifzwq (digest: /var/lib/heartbeat/crm/cib.TBhVjx)
May 28 11:29:46 node2 pengine: [10647]: notice: RecurringOp:  Start recurring monitor (15s) for scriptHa on node2
May 28 11:29:46 node2 pengine: [10647]: notice: LogActions: Leave resource failover-ip	(Started node1)
May 28 11:29:46 node2 cib: [10631]: info: Managed write_cib_contents process 27853 exited with return code 0.
May 28 11:29:46 node2 pengine: [10647]: notice: LogActions: Start scriptHa	(node2)
May 28 11:29:46 node2 pengine: [10647]: info: process_pe_message: Transition 28: PEngine Input stored in: /var/lib/pengine/pe-input-532.bz2
May 28 11:29:46 node2 pengine: [10647]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797262477s to complete
May 28 11:29:46 node2 crmd: [10635]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797262476s to complete
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797262476s to complete
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797262476s to complete
May 28 11:29:46 node2 crmd: [10635]: info: unpack_graph: Unpacked transition 28: 2 actions in 2 synapses
May 28 11:29:46 node2 crmd: [10635]: info: do_te_invoke: Processing graph 28 (ref=pe_calc-dc-1243520986-68) derived from /var/lib/pengine/pe-input-532.bz2
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_TE_INVOKE took 797262476s to complete
May 28 11:29:46 node2 crmd: [10635]: info: te_rsc_command: Initiating action 8: start scriptHa_start_0 on node2 (local)
May 28 11:29:46 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=8:28:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_start_0 )
May 28 11:29:46 node2 lrmd: [10638]: info: rsc:scriptHa: start
May 28 11:29:46 node2 lrmd: [27854]: WARN: For LSB init script, no additional parameters are needed.
May 28 11:29:46 node2 lrmd: [10638]: info: RA output: (scriptHa:start:stdout) RSC-TEST is already running, PID=27857 
May 28 11:29:46 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_start_0 (call=20, rc=0, cib-update=108, confirmed=true) complete ok
May 28 11:29:46 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_start_0 (8) confirmed on node2 (rc=0)
May 28 11:29:46 node2 crmd: [10635]: info: te_rsc_command: Initiating action 9: monitor scriptHa_monitor_15000 on node2 (local)
May 28 11:29:46 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=9:28:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_monitor_15000 )
May 28 11:29:46 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_monitor_15000 (call=21, rc=0, cib-update=109, confirmed=false) complete ok
May 28 11:29:46 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_monitor_15000 (9) confirmed on node2 (rc=0)
May 28 11:29:46 node2 crmd: [10635]: info: run_graph: ====================================================
May 28 11:29:46 node2 crmd: [10635]: notice: run_graph: Transition 28 (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-532.bz2): Complete
May 28 11:29:46 node2 crmd: [10635]: info: te_graph_trigger: Transition 28 is now complete
May 28 11:29:46 node2 crmd: [10635]: info: notify_crmd: Transition 28 status: done - <null>
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797262463s to complete
May 28 11:29:46 node2 crmd: [10635]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
May 28 11:29:46 node2 crmd: [10635]: info: do_state_transition: Starting PEngine Recheck Timer
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797262463s to complete
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797262463s to complete
May 28 11:29:46 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797262463s to complete
May 28 11:29:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:29:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:29:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:01 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_monitor_15000 (call=21, rc=7, cib-update=110, confirmed=false) complete not running
May 28 11:30:01 node2 crmd: [10635]: info: process_graph_event: Action scriptHa_monitor_15000 arrived after a completed transition
May 28 11:30:01 node2 crmd: [10635]: info: abort_transition_graph: process_graph_event:477 - Triggered transition abort (complete=1, tag=lrm_rsc_op, id=scriptHa_monitor_15000, magic=0:7;9:28:0:9a029578-94ee-450b-9007-4d83181eb556) : Inactive graph
May 28 11:30:01 node2 crmd: [10635]: WARN: update_failcount: Updating failcount for scriptHa on 8076ee23-a758-4123-9dac-cfb5edc7e76d after failed monitor: rc=7 (update=value++, time=1243521001)
May 28 11:30:01 node2 crmd: [10635]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
May 28 11:30:01 node2 crmd: [10635]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797260973s to complete
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797260973s to complete
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797260972s to complete
May 28 11:30:01 node2 crmd: [10635]: info: do_pe_invoke: Query 115: Requesting the current CIB: S_POLICY_ENGINE
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_PE_INVOKE took 797260972s to complete
May 28 11:30:01 node2 crmd: [10635]: info: do_pe_invoke_callback: Invoking the PE: ref=pe_calc-dc-1243521001-71, seq=2, quorate=1
May 28 11:30:01 node2 pengine: [10647]: WARN: unpack_resources: No STONITH resources have been defined
May 28 11:30:01 node2 pengine: [10647]: info: determine_online_status: Node node2 is online
May 28 11:30:01 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node2 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:30:01 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node2
May 28 11:30:01 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_15000 on node2 returned 7 (not running) instead of the expected value: 0 (ok)
May 28 11:30:01 node2 pengine: [10647]: WARN: unpack_rsc_op: Processing failed op scriptHa_monitor_15000 on node2: not running
May 28 11:30:01 node2 pengine: [10647]: info: determine_online_status: Node node1 is online
May 28 11:30:01 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node1 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:30:01 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node1
May 28 11:30:01 node2 pengine: [10647]: notice: native_print: failover-ip	(ocf::heartbeat:IPaddr):	Started node1
May 28 11:30:01 node2 pengine: [10647]: notice: native_print: scriptHa	(lsb:scriptHa.sh):	Started node2 FAILED
May 28 11:30:01 node2 pengine: [10647]: info: get_failcount: scriptHa has failed 3 times on node2
May 28 11:30:01 node2 pengine: [10647]: notice: common_apply_stickiness: scriptHa can fail 999997 more times on node2 before being forced off
May 28 11:30:01 node2 pengine: [10647]: notice: RecurringOp:  Start recurring monitor (15s) for scriptHa on node2
May 28 11:30:01 node2 pengine: [10647]: notice: LogActions: Leave resource failover-ip	(Started node1)
May 28 11:30:01 node2 pengine: [10647]: notice: LogActions: Recover resource scriptHa	(Started node2)
May 28 11:30:01 node2 pengine: [10647]: info: process_pe_message: Transition 29: PEngine Input stored in: /var/lib/pengine/pe-input-533.bz2
May 28 11:30:01 node2 pengine: [10647]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797260970s to complete
May 28 11:30:01 node2 crmd: [10635]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797260970s to complete
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797260970s to complete
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797260970s to complete
May 28 11:30:01 node2 crmd: [10635]: info: unpack_graph: Unpacked transition 29: 4 actions in 4 synapses
May 28 11:30:01 node2 crmd: [10635]: info: do_te_invoke: Processing graph 29 (ref=pe_calc-dc-1243521001-71) derived from /var/lib/pengine/pe-input-533.bz2
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_TE_INVOKE took 797260970s to complete
May 28 11:30:01 node2 crmd: [10635]: info: te_rsc_command: Initiating action 2: stop scriptHa_stop_0 on node2 (local)
May 28 11:30:01 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=2:29:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_stop_0 )
May 28 11:30:01 node2 lrmd: [10638]: info: rsc:scriptHa: stop
May 28 11:30:01 node2 lrmd: [27869]: WARN: For LSB init script, no additional parameters are needed.
May 28 11:30:01 node2 lrmd: [10638]: info: RA output: (scriptHa:stop:stdout) RSC-TEST is not running. 
May 28 11:30:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:01 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_monitor_15000 (call=21, rc=-2, cib-update=0, confirmed=true) Cancelled unknown exec error
May 28 11:30:01 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_stop_0 (call=22, rc=0, cib-update=116, confirmed=true) complete ok
May 28 11:30:01 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_stop_0 (2) confirmed on node2 (rc=0)
May 28 11:30:01 node2 crmd: [10635]: info: te_rsc_command: Initiating action 10: start scriptHa_start_0 on node2 (local)
May 28 11:30:01 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=10:29:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_start_0 )
May 28 11:30:01 node2 lrmd: [10638]: info: rsc:scriptHa: start
May 28 11:30:01 node2 lrmd: [27874]: WARN: For LSB init script, no additional parameters are needed.
May 28 11:30:01 node2 crmd: [10635]: info: te_pseudo_action: Pseudo action 4 fired and confirmed
May 28 11:30:01 node2 lrmd: [10638]: info: RA output: (scriptHa:start:stdout) Starting RSC-TEST.. 
May 28 11:30:01 node2 lrmd: [10638]: info: RA output: (scriptHa:start:stdout) Done. PID=27879 27884 
May 28 11:30:01 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_start_0 (call=23, rc=0, cib-update=117, confirmed=true) complete ok
May 28 11:30:01 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_start_0 (10) confirmed on node2 (rc=0)
May 28 11:30:01 node2 crmd: [10635]: info: te_rsc_command: Initiating action 1: monitor scriptHa_monitor_15000 on node2 (local)
May 28 11:30:01 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=1:29:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_monitor_15000 )
May 28 11:30:01 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 27879: binary operator expected 
May 28 11:30:01 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_monitor_15000 (call=24, rc=0, cib-update=118, confirmed=false) complete ok
May 28 11:30:01 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_monitor_15000 (1) confirmed on node2 (rc=0)
May 28 11:30:01 node2 crmd: [10635]: info: run_graph: ====================================================
May 28 11:30:01 node2 crmd: [10635]: notice: run_graph: Transition 29 (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-533.bz2): Complete
May 28 11:30:01 node2 crmd: [10635]: info: te_graph_trigger: Transition 29 is now complete
May 28 11:30:01 node2 crmd: [10635]: info: notify_crmd: Transition 29 status: done - <null>
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797260944s to complete
May 28 11:30:01 node2 crmd: [10635]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
May 28 11:30:01 node2 crmd: [10635]: info: do_state_transition: Starting PEngine Recheck Timer
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797260944s to complete
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797260944s to complete
May 28 11:30:01 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797260944s to complete
May 28 11:30:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:16 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 27879: binary operator expected 
May 28 11:30:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:31 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 27879: binary operator expected 
May 28 11:30:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:30:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:30:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:31 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 27879: binary operator expected 
May 28 11:31:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:46 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 27879: binary operator expected 
May 28 11:31:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:31:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:31:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:02 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 27879: binary operator expected 
May 28 11:32:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:17 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 27879: binary operator expected 
May 28 11:32:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:32:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:32:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:32:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:32:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:32:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 88
May 28 11:32:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797246910s to complete
May 28 11:32:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797246910s to complete
May 28 11:32:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797246910s to complete
May 28 11:32:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797246910s to complete
May 28 11:32:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:32:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797246910s to complete
May 28 11:32:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:32:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797246910s to complete
May 28 11:32:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797246910s to complete
May 28 11:32:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797246910s to complete
May 28 11:32:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797246910s to complete
May 28 11:32:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27996 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:32:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-89: Waiting on 1 outstanding join acks
May 28 11:32:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243521143-17 
May 28 11:32:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797246909s to complete
May 28 11:32:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:32:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 27997 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:32:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243521143 
May 28 11:32:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:32:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:32:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:32:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:02 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 27879: binary operator expected 
May 28 11:33:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:17 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 27879: binary operator expected 
May 28 11:33:17 node2 crmd: [10635]: info: abort_transition_graph: need_abort:59 - Triggered transition abort (complete=1) : Non-status change
May 28 11:33:17 node2 crmd: [10635]: info: need_abort: Aborting on change to admin_epoch
May 28 11:33:17 node2 crmd: [10635]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
May 28 11:33:17 node2 crmd: [10635]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
May 28 11:33:17 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797241569s to complete
May 28 11:33:17 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797241569s to complete
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: - <cib admin_epoch="0" epoch="119" num_updates="9" >
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: -   <configuration >
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: -     <resources >
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: -       <primitive id="scriptHa" >
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: -         <meta_attributes id="scriptHa-meta_attributes" >
May 28 11:33:17 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797241569s to complete
May 28 11:33:17 node2 crmd: [10635]: info: do_pe_invoke: Query 119: Requesting the current CIB: S_POLICY_ENGINE
May 28 11:33:17 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_PE_INVOKE took 797241569s to complete
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: -           <nvpair value="Started" id="scriptHa-meta_attributes-target-role" />
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: -         </meta_attributes>
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: -       </primitive>
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: -     </resources>
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: -   </configuration>
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: - </cib>
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: + <cib admin_epoch="0" epoch="120" num_updates="1" >
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: +   <configuration >
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: +     <resources >
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: +       <primitive id="scriptHa" >
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: +         <meta_attributes id="scriptHa-meta_attributes" >
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: +           <nvpair value="Stopped" id="scriptHa-meta_attributes-target-role" />
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: +         </meta_attributes>
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: +       </primitive>
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: +     </resources>
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: +   </configuration>
May 28 11:33:17 node2 cib: [10631]: info: log_data_element: cib:diff: + </cib>
May 28 11:33:17 node2 cib: [10631]: info: cib_process_request: Operation complete: op cib_modify for section resources (origin=node1/crm_resource/4, version=0.120.1): ok (rc=0)
May 28 11:33:17 node2 crmd: [10635]: info: do_pe_invoke_callback: Invoking the PE: ref=pe_calc-dc-1243521197-75, seq=2, quorate=1
May 28 11:33:17 node2 cib: [28040]: info: write_cib_contents: Archived previous version as /var/lib/heartbeat/crm/cib-38.raw
May 28 11:33:17 node2 cib: [28040]: info: write_cib_contents: Wrote version 0.120.0 of the CIB to disk (digest: a711acf48042bd19a800ec10ede588b4)
May 28 11:33:17 node2 cib: [28040]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.znfhOh (digest: /var/lib/heartbeat/crm/cib.qPcJhU)
May 28 11:33:17 node2 pengine: [10647]: WARN: unpack_resources: No STONITH resources have been defined
May 28 11:33:17 node2 pengine: [10647]: info: determine_online_status: Node node2 is online
May 28 11:33:17 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node2 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:33:17 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node2
May 28 11:33:17 node2 pengine: [10647]: info: determine_online_status: Node node1 is online
May 28 11:33:17 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node1 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:33:17 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node1
May 28 11:33:17 node2 pengine: [10647]: notice: native_print: failover-ip	(ocf::heartbeat:IPaddr):	Started node1
May 28 11:33:17 node2 pengine: [10647]: notice: native_print: scriptHa	(lsb:scriptHa.sh):	Started node2
May 28 11:33:17 node2 pengine: [10647]: info: get_failcount: scriptHa has failed 3 times on node2
May 28 11:33:17 node2 pengine: [10647]: notice: common_apply_stickiness: scriptHa can fail 999997 more times on node2 before being forced off
May 28 11:33:17 node2 pengine: [10647]: WARN: native_color: Resource scriptHa cannot run anywhere
May 28 11:33:17 node2 cib: [10631]: info: Managed write_cib_contents process 28040 exited with return code 0.
May 28 11:33:17 node2 pengine: [10647]: notice: LogActions: Leave resource failover-ip	(Started node1)
May 28 11:33:17 node2 pengine: [10647]: notice: LogActions: Stop resource scriptHa	(node2)
May 28 11:33:17 node2 pengine: [10647]: WARN: process_pe_message: Transition 30: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/pengine/pe-warn-235.bz2
May 28 11:33:17 node2 pengine: [10647]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
May 28 11:33:17 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797241564s to complete
May 28 11:33:17 node2 crmd: [10635]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
May 28 11:33:17 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797241563s to complete
May 28 11:33:17 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797241563s to complete
May 28 11:33:17 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797241563s to complete
May 28 11:33:17 node2 crmd: [10635]: info: unpack_graph: Unpacked transition 30: 2 actions in 2 synapses
May 28 11:33:17 node2 crmd: [10635]: info: do_te_invoke: Processing graph 30 (ref=pe_calc-dc-1243521197-75) derived from /var/lib/pengine/pe-warn-235.bz2
May 28 11:33:17 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_TE_INVOKE took 797241563s to complete
May 28 11:33:17 node2 crmd: [10635]: info: te_rsc_command: Initiating action 9: stop scriptHa_stop_0 on node2 (local)
May 28 11:33:17 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=9:30:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_stop_0 )
May 28 11:33:17 node2 lrmd: [10638]: info: rsc:scriptHa: stop
May 28 11:33:17 node2 lrmd: [28041]: WARN: For LSB init script, no additional parameters are needed.
May 28 11:33:17 node2 lrmd: [10638]: info: RA output: (scriptHa:stop:stdout) Stopping RSC-TEST...
May 28 11:33:17 node2 lrmd: [10638]: info: RA output: (scriptHa:stop:stderr) /etc/init.d/scriptHa.sh: line 14: [: 27879: binary operator expected /etc/init.d/scriptHa.sh: line 18: kill: (28044) - No such process 
May 28 11:33:17 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_monitor_15000 (call=24, rc=-2, cib-update=0, confirmed=true) Cancelled unknown exec error
May 28 11:33:18 node2 lrmd: [10638]: info: RA output: (scriptHa:stop:stdout) .. Done. 
May 28 11:33:18 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_stop_0 (call=25, rc=0, cib-update=120, confirmed=true) complete ok
May 28 11:33:18 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_stop_0 (9) confirmed on node2 (rc=0)
May 28 11:33:18 node2 crmd: [10635]: info: te_pseudo_action: Pseudo action 3 fired and confirmed
May 28 11:33:18 node2 crmd: [10635]: info: run_graph: ====================================================
May 28 11:33:18 node2 crmd: [10635]: notice: run_graph: Transition 30 (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-warn-235.bz2): Complete
May 28 11:33:18 node2 crmd: [10635]: info: te_graph_trigger: Transition 30 is now complete
May 28 11:33:18 node2 crmd: [10635]: info: notify_crmd: Transition 30 status: done - <null>
May 28 11:33:18 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797241457s to complete
May 28 11:33:18 node2 crmd: [10635]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
May 28 11:33:18 node2 crmd: [10635]: info: do_state_transition: Starting PEngine Recheck Timer
May 28 11:33:18 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797241457s to complete
May 28 11:33:18 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797241457s to complete
May 28 11:33:18 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797241457s to complete
May 28 11:33:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:33:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: - <cib admin_epoch="0" epoch="120" num_updates="2" >
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: -   <configuration >
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: -     <resources >
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: -       <primitive id="scriptHa" >
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: -         <meta_attributes id="scriptHa-meta_attributes" >
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: -           <nvpair value="Stopped" id="scriptHa-meta_attributes-target-role" />
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: -         </meta_attributes>
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: -       </primitive>
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: -     </resources>
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: -   </configuration>
May 28 11:33:58 node2 crmd: [10635]: info: abort_transition_graph: need_abort:59 - Triggered transition abort (complete=1) : Non-status change
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: - </cib>
May 28 11:33:58 node2 crmd: [10635]: info: need_abort: Aborting on change to admin_epoch
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: + <cib admin_epoch="0" epoch="121" num_updates="1" >
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: +   <configuration >
May 28 11:33:58 node2 crmd: [10635]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: +     <resources >
May 28 11:33:58 node2 crmd: [10635]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: +       <primitive id="scriptHa" >
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: +         <meta_attributes id="scriptHa-meta_attributes" >
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797237506s to complete
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: +           <nvpair value="Started" id="scriptHa-meta_attributes-target-role" />
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: +         </meta_attributes>
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797237506s to complete
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: +       </primitive>
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: +     </resources>
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797237506s to complete
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: +   </configuration>
May 28 11:33:58 node2 cib: [10631]: info: log_data_element: cib:diff: + </cib>
May 28 11:33:58 node2 crmd: [10635]: info: do_pe_invoke: Query 121: Requesting the current CIB: S_POLICY_ENGINE
May 28 11:33:58 node2 cib: [10631]: info: cib_process_request: Operation complete: op cib_modify for section resources (origin=node1/crm_resource/4, version=0.121.1): ok (rc=0)
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_PE_INVOKE took 797237506s to complete
May 28 11:33:58 node2 crmd: [10635]: info: do_pe_invoke_callback: Invoking the PE: ref=pe_calc-dc-1243521238-77, seq=2, quorate=1
May 28 11:33:58 node2 cib: [28049]: info: write_cib_contents: Archived previous version as /var/lib/heartbeat/crm/cib-39.raw
May 28 11:33:58 node2 pengine: [10647]: WARN: unpack_resources: No STONITH resources have been defined
May 28 11:33:58 node2 pengine: [10647]: info: determine_online_status: Node node2 is online
May 28 11:33:58 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node2 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:33:58 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node2
May 28 11:33:58 node2 pengine: [10647]: info: determine_online_status: Node node1 is online
May 28 11:33:58 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node1 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:33:58 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node1
May 28 11:33:58 node2 pengine: [10647]: notice: native_print: failover-ip	(ocf::heartbeat:IPaddr):	Started node1
May 28 11:33:58 node2 pengine: [10647]: notice: native_print: scriptHa	(lsb:scriptHa.sh):	Stopped 
May 28 11:33:58 node2 pengine: [10647]: info: get_failcount: scriptHa has failed 3 times on node2
May 28 11:33:58 node2 pengine: [10647]: notice: common_apply_stickiness: scriptHa can fail 999997 more times on node2 before being forced off
May 28 11:33:58 node2 pengine: [10647]: notice: RecurringOp:  Start recurring monitor (15s) for scriptHa on node2
May 28 11:33:58 node2 pengine: [10647]: notice: LogActions: Leave resource failover-ip	(Started node1)
May 28 11:33:58 node2 pengine: [10647]: notice: LogActions: Start scriptHa	(node2)
May 28 11:33:58 node2 cib: [28049]: info: write_cib_contents: Wrote version 0.121.0 of the CIB to disk (digest: 5add3ecabe8cc1cc0f2cc8e01bc31a23)
May 28 11:33:58 node2 pengine: [10647]: info: process_pe_message: Transition 31: PEngine Input stored in: /var/lib/pengine/pe-input-534.bz2
May 28 11:33:58 node2 cib: [28049]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.acD84p (digest: /var/lib/heartbeat/crm/cib.fSlC84)
May 28 11:33:58 node2 pengine: [10647]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797237502s to complete
May 28 11:33:58 node2 crmd: [10635]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797237502s to complete
May 28 11:33:58 node2 cib: [10631]: info: Managed write_cib_contents process 28049 exited with return code 0.
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797237501s to complete
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797237501s to complete
May 28 11:33:58 node2 crmd: [10635]: info: unpack_graph: Unpacked transition 31: 2 actions in 2 synapses
May 28 11:33:58 node2 crmd: [10635]: info: do_te_invoke: Processing graph 31 (ref=pe_calc-dc-1243521238-77) derived from /var/lib/pengine/pe-input-534.bz2
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_TE_INVOKE took 797237501s to complete
May 28 11:33:58 node2 crmd: [10635]: info: te_rsc_command: Initiating action 8: start scriptHa_start_0 on node2 (local)
May 28 11:33:58 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=8:31:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_start_0 )
May 28 11:33:58 node2 lrmd: [10638]: info: rsc:scriptHa: start
May 28 11:33:58 node2 lrmd: [28050]: WARN: For LSB init script, no additional parameters are needed.
May 28 11:33:58 node2 lrmd: [10638]: info: RA output: (scriptHa:start:stdout) RSC-TEST is already running, PID=28053 
May 28 11:33:58 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_start_0 (call=26, rc=0, cib-update=122, confirmed=true) complete ok
May 28 11:33:58 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_start_0 (8) confirmed on node2 (rc=0)
May 28 11:33:58 node2 crmd: [10635]: info: te_rsc_command: Initiating action 9: monitor scriptHa_monitor_15000 on node2 (local)
May 28 11:33:58 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=9:31:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_monitor_15000 )
May 28 11:33:58 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_monitor_15000 (call=27, rc=7, cib-update=123, confirmed=false) complete not running
May 28 11:33:58 node2 crmd: [10635]: WARN: status_from_rc: Action 9 (scriptHa_monitor_15000) on node2 failed (target: 0 vs. rc: 7): Error
May 28 11:33:58 node2 crmd: [10635]: WARN: update_failcount: Updating failcount for scriptHa on 8076ee23-a758-4123-9dac-cfb5edc7e76d after failed monitor: rc=7 (update=value++, time=1243521238)
May 28 11:33:58 node2 crmd: [10635]: info: abort_transition_graph: match_graph_event:282 - Triggered transition abort (complete=0, tag=lrm_rsc_op, id=scriptHa_monitor_15000, magic=0:7;9:31:0:9a029578-94ee-450b-9007-4d83181eb556) : Event failed
May 28 11:33:58 node2 crmd: [10635]: info: update_abort_priority: Abort priority upgraded from 0 to 1
May 28 11:33:58 node2 crmd: [10635]: info: update_abort_priority: Abort action done superceeded by restart
May 28 11:33:58 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_monitor_15000 (9) confirmed on node2 (rc=4)
May 28 11:33:58 node2 crmd: [10635]: info: run_graph: ====================================================
May 28 11:33:58 node2 crmd: [10635]: notice: run_graph: Transition 31 (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-534.bz2): Complete
May 28 11:33:58 node2 crmd: [10635]: info: te_graph_trigger: Transition 31 is now complete
May 28 11:33:58 node2 crmd: [10635]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd ]
May 28 11:33:58 node2 crmd: [10635]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797237485s to complete
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797237485s to complete
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797237484s to complete
May 28 11:33:58 node2 crmd: [10635]: info: do_pe_invoke: Query 128: Requesting the current CIB: S_POLICY_ENGINE
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_PE_INVOKE took 797237484s to complete
May 28 11:33:58 node2 crmd: [10635]: info: do_pe_invoke_callback: Invoking the PE: ref=pe_calc-dc-1243521238-80, seq=2, quorate=1
May 28 11:33:58 node2 pengine: [10647]: WARN: unpack_resources: No STONITH resources have been defined
May 28 11:33:58 node2 pengine: [10647]: info: determine_online_status: Node node2 is online
May 28 11:33:58 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node2 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:33:58 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node2
May 28 11:33:58 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_15000 on node2 returned 7 (not running) instead of the expected value: 0 (ok)
May 28 11:33:58 node2 pengine: [10647]: WARN: unpack_rsc_op: Processing failed op scriptHa_monitor_15000 on node2: not running
May 28 11:33:58 node2 pengine: [10647]: info: determine_online_status: Node node1 is online
May 28 11:33:58 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node1 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:33:58 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node1
May 28 11:33:58 node2 pengine: [10647]: notice: native_print: failover-ip	(ocf::heartbeat:IPaddr):	Started node1
May 28 11:33:58 node2 pengine: [10647]: notice: native_print: scriptHa	(lsb:scriptHa.sh):	Started node2 FAILED
May 28 11:33:58 node2 pengine: [10647]: info: get_failcount: scriptHa has failed 4 times on node2
May 28 11:33:58 node2 pengine: [10647]: notice: common_apply_stickiness: scriptHa can fail 999996 more times on node2 before being forced off
May 28 11:33:58 node2 pengine: [10647]: notice: RecurringOp:  Start recurring monitor (15s) for scriptHa on node2
May 28 11:33:58 node2 pengine: [10647]: notice: LogActions: Leave resource failover-ip	(Started node1)
May 28 11:33:58 node2 pengine: [10647]: notice: LogActions: Recover resource scriptHa	(Started node2)
May 28 11:33:58 node2 pengine: [10647]: info: process_pe_message: Transition 32: PEngine Input stored in: /var/lib/pengine/pe-input-535.bz2
May 28 11:33:58 node2 pengine: [10647]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797237482s to complete
May 28 11:33:58 node2 crmd: [10635]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797237482s to complete
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797237482s to complete
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797237482s to complete
May 28 11:33:58 node2 crmd: [10635]: info: unpack_graph: Unpacked transition 32: 4 actions in 4 synapses
May 28 11:33:58 node2 crmd: [10635]: info: do_te_invoke: Processing graph 32 (ref=pe_calc-dc-1243521238-80) derived from /var/lib/pengine/pe-input-535.bz2
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_TE_INVOKE took 797237482s to complete
May 28 11:33:58 node2 crmd: [10635]: info: te_rsc_command: Initiating action 2: stop scriptHa_stop_0 on node2 (local)
May 28 11:33:58 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=2:32:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_stop_0 )
May 28 11:33:58 node2 lrmd: [10638]: info: rsc:scriptHa: stop
May 28 11:33:58 node2 lrmd: [28060]: WARN: For LSB init script, no additional parameters are needed.
May 28 11:33:58 node2 lrmd: [10638]: info: RA output: (scriptHa:stop:stdout) RSC-TEST is not running. 
May 28 11:33:58 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_monitor_15000 (call=27, rc=-2, cib-update=0, confirmed=true) Cancelled unknown exec error
May 28 11:33:58 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_stop_0 (call=28, rc=0, cib-update=129, confirmed=true) complete ok
May 28 11:33:58 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_stop_0 (2) confirmed on node2 (rc=0)
May 28 11:33:58 node2 crmd: [10635]: info: te_rsc_command: Initiating action 10: start scriptHa_start_0 on node2 (local)
May 28 11:33:58 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=10:32:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_start_0 )
May 28 11:33:58 node2 lrmd: [10638]: info: rsc:scriptHa: start
May 28 11:33:58 node2 lrmd: [28065]: WARN: For LSB init script, no additional parameters are needed.
May 28 11:33:58 node2 crmd: [10635]: info: te_pseudo_action: Pseudo action 4 fired and confirmed
May 28 11:33:58 node2 lrmd: [10638]: info: RA output: (scriptHa:start:stdout) Starting RSC-TEST.. 
May 28 11:33:58 node2 lrmd: [10638]: info: RA output: (scriptHa:start:stdout) Done. PID=28070 28075 
May 28 11:33:58 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_start_0 (call=29, rc=0, cib-update=130, confirmed=true) complete ok
May 28 11:33:58 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_start_0 (10) confirmed on node2 (rc=0)
May 28 11:33:58 node2 crmd: [10635]: info: te_rsc_command: Initiating action 1: monitor scriptHa_monitor_15000 on node2 (local)
May 28 11:33:58 node2 crmd: [10635]: info: do_lrm_rsc_op: Performing key=1:32:0:9a029578-94ee-450b-9007-4d83181eb556 op=scriptHa_monitor_15000 )
May 28 11:33:58 node2 crmd: [10635]: info: process_lrm_event: LRM operation scriptHa_monitor_15000 (call=30, rc=0, cib-update=131, confirmed=false) complete ok
May 28 11:33:58 node2 crmd: [10635]: info: match_graph_event: Action scriptHa_monitor_15000 (1) confirmed on node2 (rc=0)
May 28 11:33:58 node2 crmd: [10635]: info: run_graph: ====================================================
May 28 11:33:58 node2 crmd: [10635]: notice: run_graph: Transition 32 (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-535.bz2): Complete
May 28 11:33:58 node2 crmd: [10635]: info: te_graph_trigger: Transition 32 is now complete
May 28 11:33:58 node2 crmd: [10635]: info: notify_crmd: Transition 32 status: done - <null>
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797237457s to complete
May 28 11:33:58 node2 crmd: [10635]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
May 28 11:33:58 node2 crmd: [10635]: info: do_state_transition: Starting PEngine Recheck Timer
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797237457s to complete
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797237457s to complete
May 28 11:33:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797237457s to complete
May 28 11:34:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:13 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:34:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:29 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:34:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:44 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:34:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:34:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:34:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:35:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:35:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:35:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:35:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:35:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 89
May 28 11:35:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797229095s to complete
May 28 11:35:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797229095s to complete
May 28 11:35:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797229095s to complete
May 28 11:35:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797229095s to complete
May 28 11:35:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:35:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797229095s to complete
May 28 11:35:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:35:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797229095s to complete
May 28 11:35:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797229095s to complete
May 28 11:35:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797229095s to complete
May 28 11:35:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797229095s to complete
May 28 11:35:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28141 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:35:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-90: Waiting on 1 outstanding join acks
May 28 11:35:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243521323-18 
May 28 11:35:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797229094s to complete
May 28 11:35:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:35:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28142 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:35:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243521323 
May 28 11:35:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:35:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:29 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:35:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:44 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:35:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:35:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:35:59 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:36:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:11 node2 cib: [10631]: info: cib_stats: Processed 31 operations (7096.00us average, 0% utilization) in the last 10min
May 28 11:36:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:14 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:36:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:36:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:36:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:29 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:37:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:44 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:37:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:37:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:37:59 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:38:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:14 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:38:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:38:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:38:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:38:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:38:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:38:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 90
May 28 11:38:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797211281s to complete
May 28 11:38:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797211281s to complete
May 28 11:38:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797211281s to complete
May 28 11:38:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797211281s to complete
May 28 11:38:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:38:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797211281s to complete
May 28 11:38:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:38:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797211281s to complete
May 28 11:38:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797211281s to complete
May 28 11:38:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797211281s to complete
May 28 11:38:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797211281s to complete
May 28 11:38:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28281 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:38:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-91: Waiting on 1 outstanding join acks
May 28 11:38:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243521503-18 
May 28 11:38:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797211280s to complete
May 28 11:38:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:38:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28282 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:38:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243521503 
May 28 11:38:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:38:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:38:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:38:59 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:39:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:15 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:39:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:30 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:39:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:45 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:39:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:39:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:39:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:00 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:40:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:15 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:40:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:40:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:40:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:00 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:41:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:15 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:41:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:41:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:41:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:41:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:41:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:41:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 91
May 28 11:41:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797193466s to complete
May 28 11:41:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797193466s to complete
May 28 11:41:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797193466s to complete
May 28 11:41:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797193466s to complete
May 28 11:41:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:41:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797193466s to complete
May 28 11:41:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:41:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797193466s to complete
May 28 11:41:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797193466s to complete
May 28 11:41:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797193466s to complete
May 28 11:41:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797193466s to complete
May 28 11:41:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28421 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:41:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-92: Waiting on 1 outstanding join acks
May 28 11:41:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243521683-18 
May 28 11:41:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797193465s to complete
May 28 11:41:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:41:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28422 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:41:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243521683 
May 28 11:41:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:41:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:30 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:41:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:45 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:41:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:41:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:41:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:15 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:42:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:45 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:42:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:42:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:42:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:00 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:43:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:15 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:43:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:30 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:43:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:45 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:43:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:43:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:43:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:01 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:44:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:44:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:44:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:44:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:44:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:44:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 92
May 28 11:44:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797175651s to complete
May 28 11:44:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797175651s to complete
May 28 11:44:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797175651s to complete
May 28 11:44:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797175651s to complete
May 28 11:44:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:44:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797175651s to complete
May 28 11:44:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:44:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797175651s to complete
May 28 11:44:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797175651s to complete
May 28 11:44:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797175651s to complete
May 28 11:44:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797175651s to complete
May 28 11:44:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28559 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:44:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-93: Waiting on 1 outstanding join acks
May 28 11:44:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243521863-18 
May 28 11:44:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797175650s to complete
May 28 11:44:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:44:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28560 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:44:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243521863 
May 28 11:44:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:44:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:44:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:44:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:01 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:45:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:16 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:45:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:31 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:45:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:46 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:45:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:45:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:45:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:01 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:46:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:46 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:46:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:46:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:46:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:01 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:47:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:16 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:47:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:47:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:47:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:47:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:47:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:47:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 93
May 28 11:47:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797157837s to complete
May 28 11:47:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797157837s to complete
May 28 11:47:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797157837s to complete
May 28 11:47:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797157837s to complete
May 28 11:47:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:47:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797157837s to complete
May 28 11:47:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:47:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797157837s to complete
May 28 11:47:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797157837s to complete
May 28 11:47:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797157837s to complete
May 28 11:47:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797157837s to complete
May 28 11:47:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28717 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:47:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-94: Waiting on 1 outstanding join acks
May 28 11:47:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243522043-18 
May 28 11:47:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797157836s to complete
May 28 11:47:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:47:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28718 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:47:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243522043 
May 28 11:47:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:47:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:31 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:47:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:46 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:47:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:47:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:47:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:46 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:48:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:48:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:48:58 node2 crmd: [10635]: info: crm_timer_popped: PEngine Recheck Timer (I_PE_CALC) just popped!
May 28 11:48:58 node2 crmd: [10635]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:48:58 node2 crmd: [10635]: WARN: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED
May 28 11:48:58 node2 crmd: [10635]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797148385s to complete
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797148385s to complete
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797148385s to complete
May 28 11:48:58 node2 crmd: [10635]: info: do_pe_invoke: Query 132: Requesting the current CIB: S_POLICY_ENGINE
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_PE_INVOKE took 797148385s to complete
May 28 11:48:58 node2 crmd: [10635]: info: do_pe_invoke_callback: Invoking the PE: ref=pe_calc-dc-1243522138-84, seq=2, quorate=1
May 28 11:48:58 node2 pengine: [10647]: WARN: unpack_resources: No STONITH resources have been defined
May 28 11:48:58 node2 pengine: [10647]: info: determine_online_status: Node node2 is online
May 28 11:48:58 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node2 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:48:58 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node2
May 28 11:48:58 node2 pengine: [10647]: info: determine_online_status: Node node1 is online
May 28 11:48:58 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node1 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 11:48:58 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node1
May 28 11:48:58 node2 pengine: [10647]: notice: native_print: failover-ip	(ocf::heartbeat:IPaddr):	Started node1
May 28 11:48:58 node2 pengine: [10647]: notice: native_print: scriptHa	(lsb:scriptHa.sh):	Started node2
May 28 11:48:58 node2 pengine: [10647]: info: get_failcount: scriptHa has failed 4 times on node2
May 28 11:48:58 node2 pengine: [10647]: notice: common_apply_stickiness: scriptHa can fail 999996 more times on node2 before being forced off
May 28 11:48:58 node2 pengine: [10647]: notice: LogActions: Leave resource failover-ip	(Started node1)
May 28 11:48:58 node2 pengine: [10647]: notice: LogActions: Leave resource scriptHa	(Started node2)
May 28 11:48:58 node2 pengine: [10647]: info: process_pe_message: Transition 33: PEngine Input stored in: /var/lib/pengine/pe-input-536.bz2
May 28 11:48:58 node2 pengine: [10647]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797148383s to complete
May 28 11:48:58 node2 crmd: [10635]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797148383s to complete
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797148383s to complete
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797148383s to complete
May 28 11:48:58 node2 crmd: [10635]: info: unpack_graph: Unpacked transition 33: 0 actions in 0 synapses
May 28 11:48:58 node2 crmd: [10635]: info: do_te_invoke: Processing graph 33 (ref=pe_calc-dc-1243522138-84) derived from /var/lib/pengine/pe-input-536.bz2
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_TE_INVOKE took 797148382s to complete
May 28 11:48:58 node2 crmd: [10635]: info: run_graph: ====================================================
May 28 11:48:58 node2 crmd: [10635]: notice: run_graph: Transition 33 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-536.bz2): Complete
May 28 11:48:58 node2 crmd: [10635]: info: te_graph_trigger: Transition 33 is now complete
May 28 11:48:58 node2 crmd: [10635]: info: notify_crmd: Transition 33 status: done - <null>
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797148382s to complete
May 28 11:48:58 node2 crmd: [10635]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
May 28 11:48:58 node2 crmd: [10635]: info: do_state_transition: Starting PEngine Recheck Timer
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797148382s to complete
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797148382s to complete
May 28 11:48:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797148382s to complete
May 28 11:49:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:02 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:49:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:17 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:49:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:49:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:49:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:17 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:50:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:50:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:50:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:50:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:50:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:50:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 94
May 28 11:50:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797140022s to complete
May 28 11:50:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797140022s to complete
May 28 11:50:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797140022s to complete
May 28 11:50:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797140022s to complete
May 28 11:50:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:50:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797140022s to complete
May 28 11:50:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:50:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797140022s to complete
May 28 11:50:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797140022s to complete
May 28 11:50:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797140022s to complete
May 28 11:50:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797140022s to complete
May 28 11:50:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28855 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:50:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-95: Waiting on 1 outstanding join acks
May 28 11:50:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243522223-19 
May 28 11:50:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797140021s to complete
May 28 11:50:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:50:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28856 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:50:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243522223 
May 28 11:50:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:50:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:32 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:50:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:47 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:50:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:50:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:50:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:02 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:51:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:17 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:51:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:51:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:51:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:02 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:52:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:17 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:52:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:32 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:52:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:52:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:52:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:02 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:53:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:53:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:53:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:53:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:53:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:53:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 95
May 28 11:53:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797122208s to complete
May 28 11:53:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797122208s to complete
May 28 11:53:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797122208s to complete
May 28 11:53:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797122208s to complete
May 28 11:53:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:53:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797122208s to complete
May 28 11:53:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:53:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797122208s to complete
May 28 11:53:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797122207s to complete
May 28 11:53:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797122207s to complete
May 28 11:53:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797122207s to complete
May 28 11:53:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28995 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:53:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-96: Waiting on 1 outstanding join acks
May 28 11:53:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243522403-19 
May 28 11:53:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797122206s to complete
May 28 11:53:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:53:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 28996 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:53:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243522403 
May 28 11:53:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:53:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:32 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:53:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:48 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:53:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:53:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:53:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:03 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:54:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:18 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:54:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:33 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:54:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:54:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:54:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:33 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:55:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:48 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:55:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:55:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:55:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:03 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:56:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:11 node2 cib: [10631]: info: cib_stats: Processed 1 operations (0.00us average, 0% utilization) in the last 10min
May 28 11:56:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:18 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:56:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:56:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:56:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:56:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:56:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:56:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 96
May 28 11:56:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797104393s to complete
May 28 11:56:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797104393s to complete
May 28 11:56:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797104393s to complete
May 28 11:56:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797104393s to complete
May 28 11:56:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:56:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797104393s to complete
May 28 11:56:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:56:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797104393s to complete
May 28 11:56:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797104393s to complete
May 28 11:56:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797104393s to complete
May 28 11:56:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797104393s to complete
May 28 11:56:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 29133 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:56:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-97: Waiting on 1 outstanding join acks
May 28 11:56:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243522583-19 
May 28 11:56:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797104392s to complete
May 28 11:56:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:56:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 29134 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:56:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243522583 
May 28 11:56:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:56:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:33 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:56:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:56:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:56:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:33 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:57:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:48 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:57:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:57:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:57:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:03 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:58:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:18 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:58:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:34 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:58:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:58:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:58:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 11:59:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 11:59:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 11:59:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 11:59:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 11:59:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 97
May 28 11:59:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797086578s to complete
May 28 11:59:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797086578s to complete
May 28 11:59:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797086578s to complete
May 28 11:59:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797086578s to complete
May 28 11:59:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 11:59:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797086578s to complete
May 28 11:59:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 11:59:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797086578s to complete
May 28 11:59:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797086578s to complete
May 28 11:59:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797086578s to complete
May 28 11:59:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797086578s to complete
May 28 11:59:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 29271 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:59:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-98: Waiting on 1 outstanding join acks
May 28 11:59:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243522763-19 
May 28 11:59:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797086577s to complete
May 28 11:59:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:59:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 29272 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 11:59:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243522763 
May 28 11:59:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 11:59:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:34 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:59:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:49 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 11:59:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 11:59:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 11:59:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:04 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:00:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:19 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:00:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:34 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:00:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:45 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:45 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:54 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:54 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:00:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:00:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:03 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:03 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:12 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:12 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:21 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:21 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:30 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:30 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:34 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:01:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:49 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:01:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:01:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:01:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:04 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:02:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:19 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:02:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 12:02:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 12:02:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 12:02:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 12:02:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 12:02:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 98
May 28 12:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797068764s to complete
May 28 12:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797068764s to complete
May 28 12:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797068764s to complete
May 28 12:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797068764s to complete
May 28 12:02:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 12:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797068764s to complete
May 28 12:02:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 12:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797068764s to complete
May 28 12:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797068764s to complete
May 28 12:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797068764s to complete
May 28 12:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797068763s to complete
May 28 12:02:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 29413 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 12:02:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-99: Waiting on 1 outstanding join acks
May 28 12:02:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243522943-19 
May 28 12:02:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797068762s to complete
May 28 12:02:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 12:02:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 29414 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 12:02:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243522943 
May 28 12:02:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 12:02:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:34 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:02:38 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:38 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:47 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:47 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:51 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:51 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:02:56 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:02:56 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:00 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:00 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:05 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:05 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:09 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:09 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:14 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:14 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:18 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:18 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:23 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:23 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:27 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:27 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:32 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:32 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:35 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:03:36 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:36 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:41 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:41 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:50 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:03:50 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:50 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:55 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:55 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:03:58 node2 crmd: [10635]: info: crm_timer_popped: PEngine Recheck Timer (I_PE_CALC) just popped!
May 28 12:03:58 node2 crmd: [10635]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 12:03:58 node2 crmd: [10635]: WARN: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED
May 28 12:03:58 node2 crmd: [10635]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
May 28 12:03:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797059310s to complete
May 28 12:03:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797059310s to complete
May 28 12:03:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797059310s to complete
May 28 12:03:58 node2 crmd: [10635]: info: do_pe_invoke: Query 133: Requesting the current CIB: S_POLICY_ENGINE
May 28 12:03:58 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_PE_INVOKE took 797059310s to complete
May 28 12:03:58 node2 crmd: [10635]: info: do_pe_invoke_callback: Invoking the PE: ref=pe_calc-dc-1243523038-85, seq=2, quorate=1
May 28 12:03:59 node2 pengine: [10647]: WARN: unpack_resources: No STONITH resources have been defined
May 28 12:03:59 node2 pengine: [10647]: info: determine_online_status: Node node2 is online
May 28 12:03:59 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node2 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 12:03:59 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node2
May 28 12:03:59 node2 pengine: [10647]: info: determine_online_status: Node node1 is online
May 28 12:03:59 node2 pengine: [10647]: info: unpack_rsc_op: scriptHa_monitor_0 on node1 returned 0 (ok) instead of the expected value: 7 (not running)
May 28 12:03:59 node2 pengine: [10647]: WARN: unpack_rsc_op: Operation scriptHa_monitor_0 found resource scriptHa active on node1
May 28 12:03:59 node2 pengine: [10647]: notice: native_print: failover-ip	(ocf::heartbeat:IPaddr):	Started node1
May 28 12:03:59 node2 pengine: [10647]: notice: native_print: scriptHa	(lsb:scriptHa.sh):	Started node2
May 28 12:03:59 node2 pengine: [10647]: info: get_failcount: scriptHa has failed 4 times on node2
May 28 12:03:59 node2 pengine: [10647]: notice: common_apply_stickiness: scriptHa can fail 999996 more times on node2 before being forced off
May 28 12:03:59 node2 pengine: [10647]: notice: LogActions: Leave resource failover-ip	(Started node1)
May 28 12:03:59 node2 pengine: [10647]: notice: LogActions: Leave resource scriptHa	(Started node2)
May 28 12:03:59 node2 pengine: [10647]: info: process_pe_message: Transition 34: PEngine Input stored in: /var/lib/pengine/pe-input-537.bz2
May 28 12:03:59 node2 pengine: [10647]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
May 28 12:03:59 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797059309s to complete
May 28 12:03:59 node2 crmd: [10635]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
May 28 12:03:59 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797059309s to complete
May 28 12:03:59 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797059309s to complete
May 28 12:03:59 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797059309s to complete
May 28 12:03:59 node2 crmd: [10635]: info: unpack_graph: Unpacked transition 34: 0 actions in 0 synapses
May 28 12:03:59 node2 crmd: [10635]: info: do_te_invoke: Processing graph 34 (ref=pe_calc-dc-1243523038-85) derived from /var/lib/pengine/pe-input-537.bz2
May 28 12:03:59 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_TE_INVOKE took 797059309s to complete
May 28 12:03:59 node2 crmd: [10635]: info: run_graph: ====================================================
May 28 12:03:59 node2 crmd: [10635]: notice: run_graph: Transition 34 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-537.bz2): Complete
May 28 12:03:59 node2 crmd: [10635]: info: te_graph_trigger: Transition 34 is now complete
May 28 12:03:59 node2 crmd: [10635]: info: notify_crmd: Transition 34 status: done - <null>
May 28 12:03:59 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_LOG    took 797059308s to complete
May 28 12:03:59 node2 crmd: [10635]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
May 28 12:03:59 node2 crmd: [10635]: info: do_state_transition: Starting PEngine Recheck Timer
May 28 12:03:59 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797059308s to complete
May 28 12:03:59 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797059308s to complete
May 28 12:03:59 node2 crmd: [10635]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797059308s to complete
May 28 12:03:59 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:03:59 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:04 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:04 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:05 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:04:08 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:08 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:13 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:13 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:17 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:17 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:20 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:04:22 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:22 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:26 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:26 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:31 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:31 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:35 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:35 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:40 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:40 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:44 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:44 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:45 node2 crmd: [10635]: info: handle_request: Current ping state: S_IDLE
May 28 12:04:48 node2 crmd: [10635]: info: handle_request: Current ping state: S_IDLE
May 28 12:04:49 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:49 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:50 node2 crmd: [10635]: info: handle_request: Current ping state: S_IDLE
May 28 12:04:53 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:53 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:04:58 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:04:58 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:02 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:02 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:07 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:07 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:11 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:11 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:16 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:16 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:20 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:05:20 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:20 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:23 node2 crmd: [10589]: ERROR: crm_timer_popped: Integration Timer (I_INTEGRATED) just popped!
May 28 12:05:23 node2 crmd: [10589]: info: crm_timer_popped: Welcomed: 1, Integrated: 0
May 28 12:05:23 node2 crmd: [10589]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_TIMER_POPPED origin=crm_timer_popped ]
May 28 12:05:23 node2 crmd: [10589]: WARN: do_state_transition: Progressed to state S_FINALIZE_JOIN after C_TIMER_POPPED
May 28 12:05:23 node2 crmd: [10589]: WARN: do_state_transition: 1 cluster nodes failed to respond to the join offer.
May 28 12:05:23 node2 crmd: [10589]: info: ghash_print_node:   Welcome reply not received from: node2 99
May 28 12:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797050949s to complete
May 28 12:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_STOP took 797050949s to complete
May 28 12:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_START took 797050949s to complete
May 28 12:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_FINALIZE took 797050949s to complete
May 28 12:05:23 node2 crmd: [10589]: WARN: do_log: FSA: Input I_ELECTION_DC from do_dc_join_finalize() received in state S_FINALIZE_JOIN
May 28 12:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_WARN   took 797050949s to complete
May 28 12:05:23 node2 crmd: [10589]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_dc_join_finalize ]
May 28 12:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_TIMER_STOP took 797050949s to complete
May 28 12:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_INTEGRATE_TIMER_START took 797050949s to complete
May 28 12:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_FINALIZE_TIMER_STOP took 797050949s to complete
May 28 12:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_ELECTION_VOTE took 797050949s to complete
May 28 12:05:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 29551 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 12:05:23 node2 crmd: [10589]: info: do_dc_join_offer_all: join-100: Waiting on 1 outstanding join acks
May 28 12:05:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="do_election_vote" t="crmd" version="3.0.1" subt="request" reference="vote-crmd-1243523123-20 
May 28 12:05:23 node2 crmd: [10589]: ERROR: do_fsa_action: Action A_DC_JOIN_OFFER_ALL took 797050948s to complete
May 28 12:05:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to <all>.crmd failed: cluster delivery failed (rc=-1) 
May 28 12:05:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Child 29552 spawned to record non-fatal assertion failure line 1375: rc == 0 
May 28 12:05:23 node2 openais[2498]: [crm  ] ERROR: send_cluster_msg_raw: Message not sent (-1): <create_request_adv origin="join_make_offer" t="crmd" version="3.0.1" subt="request" reference="join_offer-dc-1243523123 
May 28 12:05:23 node2 openais[2498]: [crm  ] WARN: route_ais_message: Sending message to node2.crmd failed: cluster delivery failed (rc=-1) 
May 28 12:05:25 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:25 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:29 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:29 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:34 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:34 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:35 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:05:39 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:39 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:43 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:43 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:48 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:48 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:50 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:05:52 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:52 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:05:57 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:05:57 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:01 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:06:01 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:04 node2 crmd: [10635]: info: handle_request: Current ping state: S_IDLE
May 28 12:06:05 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:06:06 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:06:06 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:10 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:06:10 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:11 node2 cib: [10631]: info: cib_stats: Processed 1 operations (0.00us average, 0% utilization) in the last 10min
May 28 12:06:15 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:06:15 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:19 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:06:19 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:20 node2 lrmd: [10638]: info: RA output: (scriptHa:monitor:stderr) /etc/init.d/scriptHa.sh: line 39: [: 28070: binary operator expected 
May 28 12:06:24 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:06:24 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:28 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:06:28 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:33 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:06:33 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:37 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:06:37 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:42 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:06:42 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:43 node2 crmd: [10635]: info: handle_request: Current ping state: S_IDLE
May 28 12:06:44 node2 logger: Mark:HB_REPORT:1243523249
May 28 12:06:45 node2 crmd: [10635]: info: handle_request: Current ping state: S_IDLE
May 28 12:06:45 node2 crmadmin: [29693]: info: Invoked: crmadmin -D 
May 28 12:06:45 node2 crmd: [10635]: info: handle_request: Current ping state: S_IDLE
May 28 12:06:46 node2 cibadmin: [29695]: info: Invoked: cibadmin -Ql -o nodes 
May 28 12:06:46 node2 openais[2498]: [TOTEM] The consensus timeout expired. 
May 28 12:06:46 node2 openais[2498]: [TOTEM] entering GATHER state from 3. 
May 28 12:06:47 node2 crmd: [10635]: info: handle_request: Current ping state: S_IDLE
