[ClusterLabs] pgsql Master/Slave set has no promote (all nodes are slaves)
Andrey Rubik
tirnotaure at gmail.com
Mon Oct 12 15:45:03 UTC 2015
And this is debug log after "crm_resource -N pm1 -r pgsql
--force-promote -V" command (when its fail and then stop):
Oct 12 18:37:33 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2150ca0 for uid=0 gid=0 pid=29447
id=1495b6db-a401-4c76-a1bf-e72fc4b7b01f
Oct 12 18:37:33 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-29447-14)
Oct 12 18:37:33 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [29447]
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:33 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:33 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2151200 for uid=0 gid=0 pid=29951
id=6aa8892b-9cc8-4aaa-909c-e1e63608ed36
Oct 12 18:37:33 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-29951-15)
Oct 12 18:37:33 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [29951]
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:33 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:33 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-29951-15)
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-29951-15) state:2
Oct 12 18:37:33 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-29951-15-header
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-29951-15-header
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-29951-15-header
Oct 12 18:37:33 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2151200 for uid=0 gid=0 pid=29995
id=3e3022fe-e6db-464c-966e-157885efa5ab
Oct 12 18:37:33 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-29995-15)
Oct 12 18:37:33 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [29995]
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:33 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:33 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-29995-15)
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-29995-15) state:2
Oct 12 18:37:33 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-29995-15-header
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-29995-15-header
Oct 12 18:37:33 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-29995-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2151200 for uid=0 gid=0 pid=30039
id=63f683dd-40ff-495f-96a1-9df6bc6bc6c3
Oct 12 18:37:34 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30039-15)
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30039]
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30039-15)
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30039-15) state:2
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-30039-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-30039-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-30039-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2151200 for uid=0 gid=0 pid=30083
id=f8814fa0-b6ca-46a6-973b-f39aebc97ee6
Oct 12 18:37:34 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30083-15)
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30083]
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30083-15)
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30083-15) state:2
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-30083-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-30083-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-30083-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2151200 for uid=0 gid=0 pid=30313
id=7256102e-8d1a-41a1-8b04-f0b712c6290a
Oct 12 18:37:34 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30313-15)
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30313]
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:34 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bad760 for uid=0 gid=0 pid=30313
id=c76aecdf-19fe-49ef-8c49-90e33dadf859
Oct 12 18:37:34 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-30313-11)
Oct 12 18:37:34 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [30313]
Oct 12 18:37:34 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:34 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:34 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:34 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-master-baseline[pm1] =
0000000000000000
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30313-15)
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30313-15) state:2
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-30313-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-30313-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-30313-15-header
Oct 12 18:37:34 [11333] pm1 attrd: info: attrd_peer_update:
Setting pgsql-master-baseline[pm1]: (null) -> 0000000000000000 from pm1
Oct 12 18:37:34 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:34 [11330] pm1 cib: debug: cib_process_modify:
Destroying
/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[6]
Oct 12 18:37:34 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.28 2
Oct 12 18:37:34 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.29 (null)
Oct 12 18:37:34 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=29
Oct 12 18:37:34 [11330] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']:
<nvpair id="status-1-pgsql-master-baseline" name="pgsql-master-baseline"
value="0000000000000000"/>
Oct 12 18:37:34 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.28 -> 0.19.29 (S_IDLE)
Oct 12 18:37:34 [11335] pm1 crmd: info:
abort_transition_graph: Transition aborted by
status-1-pgsql-master-baseline, pgsql-master-baseline=0000000000000000:
Transient attribute change (create cib=0.19.29,
source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'],
1)
Oct 12 18:37:34 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_IDLE cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Oct 12 18:37:34 [11335] pm1 crmd: notice:
do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [
input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Oct 12 18:37:34 [11335] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:37:34 [11335] pm1 crmd: debug: do_pe_invoke: Query
42: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:37:34 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/attrd/15, version=0.19.29)
Oct 12 18:37:34 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:34 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.29 to 0.19.28
Oct 12 18:37:34 [11335] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=42,
ref=pe_calc-dc-1444664254-33, seq=164, quorate=1
Oct 12 18:37:34 [11334] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:34 [11334] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:34 [11334] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:34 [11334] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:34 [11334] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:34 [11334] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:34 [11334] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:34 [11334] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:34 [11334] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:34 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:34 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:34 [11334] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:37:34 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:34 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:34 [11334] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:37:34 [11334] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:37:34 [11334] pm1 pengine: info: short_print:
Slaves: [ pm1 pm2 ]
Oct 12 18:37:34 [11334] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:37:34 [11334] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:34 [11334] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:34 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:1: preferring current
location (node=pm1, weight=1000000)
Oct 12 18:37:34 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:0: preferring current
location (node=pm2, weight=1000000)
Oct 12 18:37:34 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:1
Oct 12 18:37:34 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:0
Oct 12 18:37:34 [11334] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:37:34 [11334] pm1 pengine: debug: master_color: pgsql:0
master score: -1000000
Oct 12 18:37:34 [11334] pm1 pengine: debug: master_color: pgsql:1
master score: -1000000
Oct 12 18:37:34 [11334] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:37:34 [11334] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:37:34 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:37:34 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:37:34 [11334] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:37:34 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:37:34 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:37:34 [11334] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:37:34 [11334] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:37:34 [11334] pm1 pengine: info: LogActions: Leave
pgsql:0 (Slave pm2)
Oct 12 18:37:34 [11334] pm1 pengine: info: LogActions: Leave
pgsql:1 (Slave pm1)
Oct 12 18:37:34 [11334] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:37:34 [11334] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:37:34 [11334] pm1 pengine: notice: process_pe_message:
Calculated Transition 5: /var/lib/pacemaker/pengine/pe-input-100.bz2
Oct 12 18:37:34 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:34 [11335] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:34 [11335] pm1 crmd: debug: unpack_graph: Unpacked
transition 5: 0 actions in 0 synapses
Oct 12 18:37:34 [11335] pm1 crmd: info: do_te_invoke:
Processing graph 5 (ref=pe_calc-dc-1444664254-33) derived from
/var/lib/pacemaker/pengine/pe-input-100.bz2
Oct 12 18:37:34 [11335] pm1 crmd: debug: print_graph: Empty
transition graph
Oct 12 18:37:34 [11335] pm1 crmd: notice: run_graph: Transition
5 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-100.bz2): Complete
Oct 12 18:37:34 [11335] pm1 crmd: debug: te_graph_trigger:
Transition 5 is now complete
Oct 12 18:37:34 [11335] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:37:34 [11335] pm1 crmd: debug: notify_crmd:
Transition 5 status: done - <null>
Oct 12 18:37:34 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_TE_SUCCESS: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:37:34 [11335] pm1 crmd: info: do_log: FSA: Input
I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE
Oct 12 18:37:34 [11335] pm1 crmd: notice:
do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [
input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:37:34 [11335] pm1 crmd: debug:
do_state_transition: Starting PEngine Recheck Timer
Oct 12 18:37:34 [11335] pm1 crmd: debug: crm_timer_start:
Started PEngine Recheck Timer (I_PE_CALC:900000ms), src=67
Oct 12 18:37:34 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-30313-11)
Oct 12 18:37:34 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-30313-11) state:2
Oct 12 18:37:34 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:34 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-30313-11-header
Oct 12 18:37:34 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-30313-11-header
Oct 12 18:37:34 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-30313-11-header
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2366430 for uid=0 gid=0 pid=30572
id=6c6cdfe8-787d-4362-9591-4271365e2498
Oct 12 18:37:34 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30572-15)
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30572]
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30572-15)
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30572-15) state:2
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-30572-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-30572-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-30572-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2366430 for uid=0 gid=0 pid=30589
id=7738ff5c-88ab-402e-a0e2-53afbcd84b5b
Oct 12 18:37:34 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30589-15)
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30589]
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:34 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:34 [11330] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/configuration/nodes//node[@id='1']//instance_attributes//nvpair[@name='pgsql-data-status']
(/cib/configuration/nodes/node[1]/instance_attributes/nvpair)
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30589-15)
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30589-15) state:2
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-30589-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-30589-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-30589-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2366430 for uid=0 gid=0 pid=30606
id=0f40b143-07f7-45df-80c6-303c74ae3901
Oct 12 18:37:34 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30606-15)
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30606]
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:34 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:34 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bad760 for uid=0 gid=0 pid=30606
id=c119dcad-e6df-43b4-8ca3-f89631a4cb5a
Oct 12 18:37:34 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-30606-11)
Oct 12 18:37:34 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [30606]
Oct 12 18:37:34 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:34 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:34 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:34 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting master-pgsql:0[pm1] = 1000
Oct 12 18:37:34 [11333] pm1 attrd: info: attrd_peer_update:
Setting master-pgsql:0[pm1]: (null) -> 1000 from pm1
Oct 12 18:37:34 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30606-15)
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30606-15) state:2
Oct 12 18:37:34 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-30606-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-30606-15-header
Oct 12 18:37:34 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-30606-15-header
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.29 2
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.30 (null)
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=30
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']:
<nvpair id="status-1-master-pgsql.0" name="master-pgsql:0" value="1000"/>
Oct 12 18:37:35 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.29 -> 0.19.30 (S_IDLE)
Oct 12 18:37:35 [11335] pm1 crmd: info:
abort_transition_graph: Transition aborted by
status-1-master-pgsql.0, master-pgsql:0=1000: Transient attribute change
(create cib=0.19.30, source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'],
1)
Oct 12 18:37:35 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_IDLE cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Oct 12 18:37:35 [11335] pm1 crmd: notice:
do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [
input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Oct 12 18:37:35 [11335] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:37:35 [11335] pm1 crmd: debug: do_pe_invoke: Query
43: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:37:35 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.30 to 0.19.29
Oct 12 18:37:35 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/attrd/16, version=0.19.30)
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11335] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=43,
ref=pe_calc-dc-1444664255-34, seq=164, quorate=1
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:35 [11334] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:35 [11334] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:35 [11334] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:35 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:35 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:35 [11334] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:37:35 [11334] pm1 pengine: info: short_print:
Slaves: [ pm1 pm2 ]
Oct 12 18:37:35 [11334] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:37:35 [11334] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:35 [11334] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:35 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:1: preferring current
location (node=pm1, weight=1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:0: preferring current
location (node=pm2, weight=1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:1
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:0
Oct 12 18:37:35 [11334] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:37:35 [11334] pm1 pengine: debug: master_color: pgsql:0
master score: -1000000
Oct 12 18:37:35 [11334] pm1 pengine: debug: master_color: pgsql:1
master score: -1000000
Oct 12 18:37:35 [11334] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:37:35 [11334] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:37:35 [11334] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:37:35 [11334] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:37:35 [11334] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
pgsql:0 (Slave pm2)
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
pgsql:1 (Slave pm1)
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:37:35 [11334] pm1 pengine: notice: process_pe_message:
Calculated Transition 6: /var/lib/pacemaker/pengine/pe-input-101.bz2
Oct 12 18:37:35 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:35 [11335] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:35 [11335] pm1 crmd: debug: unpack_graph: Unpacked
transition 6: 0 actions in 0 synapses
Oct 12 18:37:35 [11335] pm1 crmd: info: do_te_invoke:
Processing graph 6 (ref=pe_calc-dc-1444664255-34) derived from
/var/lib/pacemaker/pengine/pe-input-101.bz2
Oct 12 18:37:35 [11335] pm1 crmd: debug: print_graph: Empty
transition graph
Oct 12 18:37:35 [11335] pm1 crmd: notice: run_graph: Transition
6 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-101.bz2): Complete
Oct 12 18:37:35 [11335] pm1 crmd: debug: te_graph_trigger:
Transition 6 is now complete
Oct 12 18:37:35 [11335] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:37:35 [11335] pm1 crmd: debug: notify_crmd:
Transition 6 status: done - <null>
Oct 12 18:37:35 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_TE_SUCCESS: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:37:35 [11335] pm1 crmd: info: do_log: FSA: Input
I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE
Oct 12 18:37:35 [11335] pm1 crmd: notice:
do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [
input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:37:35 [11335] pm1 crmd: debug:
do_state_transition: Starting PEngine Recheck Timer
Oct 12 18:37:35 [11335] pm1 crmd: debug: crm_timer_start:
Started PEngine Recheck Timer (I_PE_CALC:900000ms), src=69
Oct 12 18:37:35 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-30606-11)
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-30606-11) state:2
Oct 12 18:37:35 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-30606-11-header
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-30606-11-header
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-30606-11-header
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2366430 for uid=0 gid=0 pid=30642
id=00a72ea7-6153-42fe-86c6-a638e1d9a7c0
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30642-15)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30642]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30642-15)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30642-15) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-30642-15-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-30642-15-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-30642-15-header
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2366430 for uid=0 gid=0 pid=30658
id=2efe1ab2-5c40-4ede-bca1-9b65ba179c57
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30658-15)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30658]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/status//node_state[@id='1']//transient_attributes//nvpair[@name='pgsql-status']
(/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3])
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30658-15)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30658-15) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-30658-15-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-30658-15-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-30658-15-header
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x2366430 for uid=0 gid=0 pid=30690
id=9ab61d89-e022-4e43-9580-453b3440c09f
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30690-15)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30690]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bad760 for uid=0 gid=0 pid=30690
id=0fd44a22-5351-4f4e-bd4e-2898ecaebff1
Oct 12 18:37:35 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-30690-11)
Oct 12 18:37:35 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [30690]
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-status[pm1] = PRI
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30690-15)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30690-15) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-30690-15-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-30690-15-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-30690-15-header
Oct 12 18:37:35 [11333] pm1 attrd: info: attrd_peer_update:
Setting pgsql-status[pm1]: HS:alone -> PRI from pm1
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.30 2
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.31 (null)
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=31
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-pgsql-status']:
@value=PRI
Oct 12 18:37:35 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.31 to 0.19.30
Oct 12 18:37:35 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.30 -> 0.19.31 (S_IDLE)
Oct 12 18:37:35 [11335] pm1 crmd: info:
abort_transition_graph: Transition aborted by status-1-pgsql-status,
pgsql-status=PRI: Transient attribute change (modify cib=0.19.31,
source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-pgsql-status'],
1)
Oct 12 18:37:35 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_IDLE cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Oct 12 18:37:35 [11335] pm1 crmd: notice:
do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [
input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Oct 12 18:37:35 [11335] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:37:35 [11335] pm1 crmd: debug: do_pe_invoke: Query
44: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:37:35 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/attrd/17, version=0.19.31)
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11335] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=44,
ref=pe_calc-dc-1444664255-35, seq=164, quorate=1
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:35 [11334] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:35 [11334] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:35 [11334] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:35 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:35 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:35 [11334] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:37:35 [11334] pm1 pengine: info: short_print:
Slaves: [ pm1 pm2 ]
Oct 12 18:37:35 [11334] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:37:35 [11334] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:35 [11334] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:35 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:1: preferring current
location (node=pm1, weight=1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:0: preferring current
location (node=pm2, weight=1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:1
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:0
Oct 12 18:37:35 [11334] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:37:35 [11334] pm1 pengine: debug: master_color: pgsql:0
master score: -1000000
Oct 12 18:37:35 [11334] pm1 pengine: debug: master_color: pgsql:1
master score: -1000000
Oct 12 18:37:35 [11334] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:37:35 [11334] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:37:35 [11334] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:37:35 [11334] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:37:35 [11334] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
pgsql:0 (Slave pm2)
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
pgsql:1 (Slave pm1)
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:37:35 [11334] pm1 pengine: notice: process_pe_message:
Calculated Transition 7: /var/lib/pacemaker/pengine/pe-input-102.bz2
Oct 12 18:37:35 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:35 [11335] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:35 [11335] pm1 crmd: debug: unpack_graph: Unpacked
transition 7: 0 actions in 0 synapses
Oct 12 18:37:35 [11335] pm1 crmd: info: do_te_invoke:
Processing graph 7 (ref=pe_calc-dc-1444664255-35) derived from
/var/lib/pacemaker/pengine/pe-input-102.bz2
Oct 12 18:37:35 [11335] pm1 crmd: debug: print_graph: Empty
transition graph
Oct 12 18:37:35 [11335] pm1 crmd: notice: run_graph: Transition
7 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-102.bz2): Complete
Oct 12 18:37:35 [11335] pm1 crmd: debug: te_graph_trigger:
Transition 7 is now complete
Oct 12 18:37:35 [11335] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:37:35 [11335] pm1 crmd: debug: notify_crmd:
Transition 7 status: done - <null>
Oct 12 18:37:35 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_TE_SUCCESS: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:37:35 [11335] pm1 crmd: info: do_log: FSA: Input
I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE
Oct 12 18:37:35 [11335] pm1 crmd: notice:
do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [
input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:37:35 [11335] pm1 crmd: debug:
do_state_transition: Starting PEngine Recheck Timer
Oct 12 18:37:35 [11335] pm1 crmd: debug: crm_timer_start:
Started PEngine Recheck Timer (I_PE_CALC:900000ms), src=71
Oct 12 18:37:35 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-30690-11)
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-30690-11) state:2
Oct 12 18:37:35 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-30690-11-header
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-30690-11-header
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-30690-11-header
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-29447-14)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-29447-14) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-29447-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-29447-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-29447-14-header
Oct 12 18:37:35 [11332] pm1 lrmd: debug:
recurring_action_timer: Scheduling another invocation of
pgsql_monitor_4000
Oct 12 18:37:35 [30735] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:35 [30735] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-30735-24-header
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-30735-24-header
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-30735-24-header
Oct 12 18:37:35 [30735] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-30735-24-header
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-30735-24-header
Oct 12 18:37:35 [30735] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-30735-24-header
Oct 12 18:37:35 [30735] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
pgsql(pgsql)[30701]: 2015/10/12_18:37:35 DEBUG: PostgreSQL is running
as a primary.
Oct 12 18:37:35 [30808] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=30808
id=c9e1ae91-fa04-4ae9-8d60-ee905cb55cc0
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30808-14)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30808]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30808] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30808] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30808] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30808] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [30808] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:35 [30808] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:35 [30808] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:35 [30808] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:35 [30808] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:35 [30808] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:35 [30808] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:35 [30808] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:35 [30808] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:35 [30808] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:35 [30808] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:35 [30808] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30808-14)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30808-14) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-30808-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-30808-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-30808-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=30810
id=be748484-014c-4f18-9969-f24bcec89d89
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30810-14)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30810]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:35 [30810] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/configuration/nodes//node[@id='1']//instance_attributes//nvpair[@name='pgsql-data-status']
(/cib/configuration/nodes/node[1]/instance_attributes/nvpair)
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
find_nvpair_attr_delegate: Match <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:35 [30810] pm1 crm_attribute: info: main: Read
pgsql-data-status=LATEST
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-30810-14-header
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-30810-14-header
Oct 12 18:37:35 [30810] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-30810-14-header
Oct 12 18:37:35 [30810] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30810-14)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30810-14) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-30810-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-30810-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-30810-14-header
Oct 12 18:37:35 [30817] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=30817
id=1851e4e9-28ef-4a3d-bc63-4f6d0b606c4a
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30817-14)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30817]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30817] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30817] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30817] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30817] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [30817] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:35 [30817] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:35 [30817] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:35 [30817] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:35 [30817] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:35 [30817] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:35 [30817] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:35 [30817] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:35 [30817] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:35 [30817] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:35 [30817] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:35 [30817] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30817-14)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30817-14) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-30817-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-30817-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-30817-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=30819
id=770b8852-6b8a-47a3-b11f-5ddf57086c2e
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30819-14)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30819]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:35 [30819] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/status//node_state[@id='1']//transient_attributes//nvpair[@name='pgsql-status']
(/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3])
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
find_nvpair_attr_delegate: Match <nvpair id="status-1-pgsql-status"
name="pgsql-status" value="PRI"/>
Oct 12 18:37:35 [30819] pm1 crm_attribute: info: main: Read
pgsql-status=PRI
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-30819-14-header
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-30819-14-header
Oct 12 18:37:35 [30819] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-30819-14-header
Oct 12 18:37:35 [30819] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30819-14)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30819-14) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-30819-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-30819-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-30819-14-header
Oct 12 18:37:35 [30834] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=30834
id=4144d109-8d67-4f3a-8d69-05c32616bc06
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30834-14)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30834]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30834] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30834] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30834] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30834] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [30834] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:35 [30834] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:35 [30834] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:35 [30834] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:35 [30834] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:35 [30834] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:35 [30834] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:35 [30834] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:35 [30834] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:35 [30834] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:35 [30834] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:35 [30834] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30834-14)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30834-14) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-30834-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-30834-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-30834-14-header
Oct 12 18:37:35 [30841] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=30841
id=0f3f6555-2721-4c7c-9031-1d62ad71b438
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30841-14)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30841]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30841] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30841] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30841] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30841] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [30841] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:35 [30841] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:35 [30841] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:35 [30841] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:35 [30841] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:35 [30841] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:35 [30841] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:35 [30841] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:35 [30841] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:35 [30841] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:35 [30841] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:35 [30841] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30841-14)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30841-14) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-30841-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-30841-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-30841-14-header
pgsql(pgsql)[30701]: 2015/10/12_18:37:35 DEBUG: pg1 is already in
async mode.
Oct 12 18:37:35 [30853] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=30853
id=62ea9055-8577-4b92-8d9f-975e673aa2c3
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30853-14)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30853]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30853] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30853] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30853] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30853] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [30853] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:35 [30853] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:35 [30853] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:35 [30853] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:35 [30853] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:35 [30853] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:35 [30853] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:35 [30853] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:35 [30853] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:35 [30853] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:35 [30853] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:35 [30853] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30853-14)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30853-14) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-30853-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-30853-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-30853-14-header
Oct 12 18:37:35 [30860] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=30860
id=4fc064c7-1e09-4f98-bfa1-29cfc97fd550
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-30860-14)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [30860]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30860] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30860] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30860] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [30860] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [30860] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:35 [30860] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:35 [30860] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:35 [30860] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:35 [30860] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:35 [30860] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:35 [30860] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:35 [30860] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:35 [30860] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:35 [30860] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:35 [30860] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:35 [30860] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-30860-14)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-30860-14) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-30860-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-30860-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-30860-14-header
pgsql(pgsql)[30701]: 2015/10/12_18:37:35 DEBUG: pg2 is already in
async mode.
Oct 12 18:37:35 [11332] pm1 lrmd: debug: operation_finished:
pgsql_monitor_4000:30701 - exited with rc=8
Oct 12 18:37:35 [11332] pm1 lrmd: debug: operation_finished:
pgsql_monitor_4000:30701:stderr [ -- empty -- ]
Oct 12 18:37:35 [11332] pm1 lrmd: debug: operation_finished:
pgsql_monitor_4000:30701:stdout [ -- empty -- ]
Oct 12 18:37:35 [11332] pm1 lrmd: debug: log_finished: finished
- rsc:pgsql action:monitor call_id:17 pid:30701 exit-code:8
exec-time:0ms queue-time:0ms
Oct 12 18:37:35 [11335] pm1 crmd: debug:
create_operation_update: do_update_resource: Updating resource pgsql
after monitor op complete (interval=4000)
Oct 12 18:37:35 [11335] pm1 crmd: info: process_lrm_event:
Operation pgsql_monitor_4000: master (node=pm1, call=17, rc=8,
cib-update=45, confirmed=false)
Oct 12 18:37:35 [11335] pm1 crmd: debug:
update_history_cache: Updating history for 'pgsql' with monitor op
Oct 12 18:37:35 [11330] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/45)
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.31 2
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.32 (null)
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=32
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='pgsql']:
<lrm_rsc_op id="pgsql_last_failure_0" operation_key="pgsql_monitor_4000"
operation="monitor" crm-debug-origin="do_update_resource"
crm_feature_set="3.0.10"
transition-key="10:3:0:27561a4e-40a7-4815-98c0-df6b485550dc"
transition-magic="0:8;10:3:0:27561a4e-40a7-4815-98c0-df6b485550dc"
on_node="pm1" call-id="17" rc-code="8" op-status="0" interval="4000" la
Oct 12 18:37:35 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.32 to 0.19.31
Oct 12 18:37:35 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.31 -> 0.19.32 (S_IDLE)
Oct 12 18:37:35 [11335] pm1 crmd: info:
abort_transition_graph: Transition aborted by pgsql_monitor_4000
'create' on pm1: Old event
(magic=0:8;10:3:0:27561a4e-40a7-4815-98c0-df6b485550dc, cib=0.19.32,
source=process_graph_event:593, 1)
Oct 12 18:37:35 [11335] pm1 crmd: info: update_failcount:
Updating failcount for pgsql on pm1 after failed monitor: rc=8
(update=value++, time=1444664255)
Oct 12 18:37:35 [11335] pm1 crmd: debug:
attrd_update_delegate: Sent update: fail-count-pgsql=value++ for pm1
Oct 12 18:37:35 [11335] pm1 crmd: debug:
attrd_update_delegate: Sent update: last-failure-pgsql=1444664255 for pm1
Oct 12 18:37:35 [11335] pm1 crmd: info:
process_graph_event: Detected action (3.10)
pgsql_monitor_4000.17=master: failed
Oct 12 18:37:35 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_IDLE cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Oct 12 18:37:35 [11335] pm1 crmd: notice:
do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [
input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Oct 12 18:37:35 [11335] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:37:35 [11335] pm1 crmd: debug: do_pe_invoke: Query
46: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:37:35 [11333] pm1 attrd: info:
attrd_client_update: Expanded fail-count-pgsql=value++ to 1
Oct 12 18:37:35 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting fail-count-pgsql[pm1] = 1
Oct 12 18:37:35 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting last-failure-pgsql[pm1] = 1444664255
Oct 12 18:37:35 [11333] pm1 attrd: info: attrd_peer_update:
Setting fail-count-pgsql[pm1]: (null) -> 1 from pm1
Oct 12 18:37:35 [11333] pm1 attrd: info: attrd_peer_update:
Setting last-failure-pgsql[pm1]: (null) -> 1444664255 from pm1
Oct 12 18:37:35 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/45, version=0.19.32)
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.32 2
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.33 (null)
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=33
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']:
<nvpair id="status-1-fail-count-pgsql" name="fail-count-pgsql" value="1"/>
Oct 12 18:37:35 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/attrd/18, version=0.19.33)
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.33 2
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.34 (null)
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=34
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']:
<nvpair id="status-1-last-failure-pgsql" name="last-failure-pgsql"
value="1444664255"/>
Oct 12 18:37:35 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/attrd/19, version=0.19.34)
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.33 to 0.19.32
Oct 12 18:37:35 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.34 to 0.19.33
Oct 12 18:37:35 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.32 -> 0.19.33 (S_POLICY_ENGINE)
Oct 12 18:37:35 [11335] pm1 crmd: info:
abort_transition_graph: Transition aborted by
status-1-fail-count-pgsql, fail-count-pgsql=1: Transient attribute
change (create cib=0.19.33, source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'],
1)
Oct 12 18:37:35 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.33 -> 0.19.34 (S_POLICY_ENGINE)
Oct 12 18:37:35 [11335] pm1 crmd: info:
abort_transition_graph: Transition aborted by
status-1-last-failure-pgsql, last-failure-pgsql=1444664255: Transient
attribute change (create cib=0.19.34, source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'],
1)
Oct 12 18:37:35 [11335] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=46,
ref=pe_calc-dc-1444664255-36, seq=164, quorate=1
Oct 12 18:37:35 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_POLICY_ENGINE cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Oct 12 18:37:35 [11335] pm1 crmd: debug: do_pe_invoke: Query
47: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:37:35 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_POLICY_ENGINE cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Oct 12 18:37:35 [11335] pm1 crmd: debug: do_pe_invoke: Query
48: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:35 [11334] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:35 [11334] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:35 [11334] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:35 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:35 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:35 [11334] pm1 pengine: debug:
determine_op_status: pgsql_monitor_4000 on pm1 returned 'master' (8)
instead of the expected value: 'ok' (0)
Oct 12 18:37:35 [11334] pm1 pengine: warning:
unpack_rsc_op_failure: Processing failed op monitor for pgsql:1 on
pm1: master (8)
Oct 12 18:37:35 [11334] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:37:35 [11334] pm1 pengine: info: native_print:
pgsql (ocf::heartbeat:pgsql): FAILED Master pm1
Oct 12 18:37:35 [11334] pm1 pengine: info: short_print:
Slaves: [ pm2 ]
Oct 12 18:37:35 [11334] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:37:35 [11334] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:35 [11334] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:35 [11334] pm1 pengine: info: get_failcount_full:
pgsql:0 has failed 1 times on pm1
Oct 12 18:37:35 [11334] pm1 pengine: info:
common_apply_stickiness: msPostgresql can fail 1 more times on pm1
before being forced off
Oct 12 18:37:35 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:1: preferring current
location (node=pm1, weight=1000000)
Oct 12 18:37:35 [11334] pm1 pengine: info: get_failcount_full:
pgsql:1 has failed 1 times on pm1
Oct 12 18:37:35 [11334] pm1 pengine: info:
common_apply_stickiness: msPostgresql can fail 1 more times on pm1
before being forced off
Oct 12 18:37:35 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:0: preferring current
location (node=pm2, weight=1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:0
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:1
Oct 12 18:37:35 [11334] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:37:35 [11334] pm1 pengine: debug: master_color: pgsql:1
master score: -1000000
Oct 12 18:37:35 [11334] pm1 pengine: debug: master_color: pgsql:0
master score: -1000000
Oct 12 18:37:35 [11334] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:37:35 [11334] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:37:35 [11334] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:37:35 [11334] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:37:35 [11334] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
pgsql:0 (Slave pm2)
Oct 12 18:37:35 [11334] pm1 pengine: notice: LogActions: Demote
pgsql:1 (Master -> Slave pm1)
Oct 12 18:37:35 [11334] pm1 pengine: notice: LogActions: Recover
pgsql:1 (Master pm1)
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: notice: process_pe_message:
Calculated Transition 8: /var/lib/pacemaker/pengine/pe-input-103.bz2
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11335] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=48,
ref=pe_calc-dc-1444664255-37, seq=164, quorate=1
Oct 12 18:37:35 [11335] pm1 crmd: info: handle_response:
pe_calc calculation pe_calc-dc-1444664255-36 is obsolete
Oct 12 18:37:35 [11334] pm1 pengine: info: process_pe_message:
Input has not changed since last time, not saving to disk
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:35 [11334] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:35 [11334] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:35 [11334] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:35 [11334] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:35 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:35 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:35 [11334] pm1 pengine: debug:
determine_op_status: pgsql_monitor_4000 on pm1 returned 'master' (8)
instead of the expected value: 'ok' (0)
Oct 12 18:37:35 [11334] pm1 pengine: warning:
unpack_rsc_op_failure: Processing failed op monitor for pgsql:1 on
pm1: master (8)
Oct 12 18:37:35 [11334] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:37:35 [11334] pm1 pengine: info: native_print:
pgsql (ocf::heartbeat:pgsql): FAILED Master pm1
Oct 12 18:37:35 [11334] pm1 pengine: info: short_print:
Slaves: [ pm2 ]
Oct 12 18:37:35 [11334] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:37:35 [11334] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:35 [11334] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:35 [11334] pm1 pengine: info: get_failcount_full:
pgsql:0 has failed 1 times on pm1
Oct 12 18:37:35 [11334] pm1 pengine: info:
common_apply_stickiness: msPostgresql can fail 1 more times on pm1
before being forced off
Oct 12 18:37:35 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:1: preferring current
location (node=pm1, weight=1000000)
Oct 12 18:37:35 [11334] pm1 pengine: info: get_failcount_full:
pgsql:1 has failed 1 times on pm1
Oct 12 18:37:35 [11334] pm1 pengine: info:
common_apply_stickiness: msPostgresql can fail 1 more times on pm1
before being forced off
Oct 12 18:37:35 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:0: preferring current
location (node=pm2, weight=1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:0
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:1
Oct 12 18:37:35 [11334] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:37:35 [11334] pm1 pengine: debug: master_color: pgsql:1
master score: -1000000
Oct 12 18:37:35 [11334] pm1 pengine: debug: master_color: pgsql:0
master score: -1000000
Oct 12 18:37:35 [11334] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:37:35 [11334] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:37:35 [11334] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:37:35 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:37:35 [11334] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:37:35 [11334] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
pgsql:0 (Slave pm2)
Oct 12 18:37:35 [11334] pm1 pengine: notice: LogActions: Demote
pgsql:1 (Master -> Slave pm1)
Oct 12 18:37:35 [11334] pm1 pengine: notice: LogActions: Recover
pgsql:1 (Master pm1)
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:37:35 [11334] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:35 [11334] pm1 pengine: notice: process_pe_message:
Calculated Transition 9: /var/lib/pacemaker/pengine/pe-input-103.bz2
Oct 12 18:37:35 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:35 [11335] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:35 [11335] pm1 crmd: debug: unpack_graph: Unpacked
transition 9: 33 actions in 33 synapses
Oct 12 18:37:35 [11335] pm1 crmd: info: do_te_invoke:
Processing graph 9 (ref=pe_calc-dc-1444664255-37) derived from
/var/lib/pacemaker/pengine/pe-input-103.bz2
Oct 12 18:37:35 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 32 fired and confirmed
Oct 12 18:37:35 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=0, Pending=0, Fired=1, Skipped=0, Incomplete=32,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:35 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 51: notify pgsql_pre_notify_demote_0 on pm2
Oct 12 18:37:35 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 53: notify pgsql_pre_notify_demote_0 on pm1 (local)
Oct 12 18:37:35 [11335] pm1 crmd: info: do_lrm_rsc_op:
Performing key=53:9:0:27561a4e-40a7-4815-98c0-df6b485550dc op=pgsql_notify_0
Oct 12 18:37:35 [11332] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
a803ca7c-a9b1-490f-9ab6-5c18bb81be3f: rc=18, reply=1, notify=0, exit=4203648
Oct 12 18:37:35 [11332] pm1 lrmd: info: log_execute: executing
- rsc:pgsql action:notify call_id:18
Oct 12 18:37:35 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=1, Pending=2, Fired=2, Skipped=0, Incomplete=30,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:35 [30901] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:35 [30901] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-30901-24-header
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-30901-24-header
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-30901-24-header
Oct 12 18:37:35 [30901] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-30901-24-header
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-30901-24-header
Oct 12 18:37:35 [30901] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-30901-24-header
Oct 12 18:37:35 [30901] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
pgsql(pgsql)[30867]: 2015/10/12_18:37:35 DEBUG: notify: pre for demote
Oct 12 18:37:35 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:30867 - exited with rc=0
Oct 12 18:37:35 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:30867:stderr [ -- empty -- ]
Oct 12 18:37:35 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:30867:stdout [ -- empty -- ]
Oct 12 18:37:35 [11332] pm1 lrmd: info: log_finished: finished
- rsc:pgsql action:notify call_id:18 pid:30867 exit-code:0
exec-time:94ms queue-time:0ms
Oct 12 18:37:35 [11335] pm1 crmd: debug:
create_operation_update: send_direct_ack: Updating resource pgsql
after notify op complete (interval=0)
Oct 12 18:37:35 [11335] pm1 crmd: debug: send_direct_ack:
ACK'ing resource op pgsql_notify_0 from
53:9:0:27561a4e-40a7-4815-98c0-df6b485550dc: lrm_invoke-lrmd-1444664255-40
Oct 12 18:37:35 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664255-40 from pm1
Oct 12 18:37:35 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (53) confirmed on pm1 (rc=0)
Oct 12 18:37:35 [11335] pm1 crmd: notice: process_lrm_event:
Operation pgsql_notify_0: ok (node=pm1, call=18, rc=0, cib-update=0,
confirmed=true)
Oct 12 18:37:35 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=2, Pending=1, Fired=0, Skipped=0, Incomplete=30,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:35 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664255-8 from pm2
Oct 12 18:37:35 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (51) confirmed on pm2 (rc=0)
Oct 12 18:37:35 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 33 fired and confirmed
Oct 12 18:37:35 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 30 fired and confirmed
Oct 12 18:37:35 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=3, Pending=0, Fired=2, Skipped=0, Incomplete=28,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:35 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 10: demote pgsql_demote_0 on pm1 (local)
Oct 12 18:37:35 [11335] pm1 crmd: debug:
stop_recurring_action_by_rsc: Cancelling op 17 for pgsql (pgsql:17)
Oct 12 18:37:35 [11335] pm1 crmd: debug: cancel_op: Cancelling
op 17 for pgsql (pgsql:17)
Oct 12 18:37:35 [11332] pm1 lrmd: info:
cancel_recurring_action: Cancelling operation pgsql_monitor_4000
Oct 12 18:37:35 [11332] pm1 lrmd: debug: log_finished: finished
- rsc:pgsql action:monitor call_id:17 exit-code:8 exec-time:0ms
queue-time:0ms
Oct 12 18:37:35 [11332] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_cancel operation from
a803ca7c-a9b1-490f-9ab6-5c18bb81be3f: rc=0, reply=1, notify=0, exit=4203648
Oct 12 18:37:35 [11335] pm1 crmd: debug: cancel_op: Op 17
for pgsql (pgsql:17): cancelled
Oct 12 18:37:35 [11335] pm1 crmd: debug: do_lrm_rsc_op: Stopped
0 recurring operations in preparation for pgsql_demote_0
Oct 12 18:37:35 [11335] pm1 crmd: info: do_lrm_rsc_op:
Performing key=10:9:0:27561a4e-40a7-4815-98c0-df6b485550dc op=pgsql_demote_0
Oct 12 18:37:35 [11332] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
a803ca7c-a9b1-490f-9ab6-5c18bb81be3f: rc=20, reply=1, notify=0, exit=4203648
Oct 12 18:37:35 [11332] pm1 lrmd: info: log_execute: executing
- rsc:pgsql action:demote call_id:20
Oct 12 18:37:35 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=5, Pending=1, Fired=1, Skipped=0, Incomplete=27,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:35 [11335] pm1 crmd: info: process_lrm_event:
Operation pgsql_monitor_4000: Cancelled (node=pm1, call=17, confirmed=true)
Oct 12 18:37:35 [11335] pm1 crmd: debug:
update_history_cache: Updating history for 'pgsql' with monitor op
Oct 12 18:37:35 [30989] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:35 [30989] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-30989-24-header
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-30989-24-header
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-30989-24-header
Oct 12 18:37:35 [30989] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-30989-24-header
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-30989-24-header
Oct 12 18:37:35 [30989] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-30989-24-header
Oct 12 18:37:35 [30989] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:35 [31040] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:35 [31040] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31040-24-header
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31040-24-header
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31040-24-header
Oct 12 18:37:35 [31040] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31040-24-header
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31040-24-header
Oct 12 18:37:35 [31040] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31040-24-header
Oct 12 18:37:35 [31040] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=31042
id=e5899ab2-15fb-49dc-96e6-a69969618dfc
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31042-14)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31042]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:35 [31042] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:35 [31042] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:37:35 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bab000 for uid=0 gid=0 pid=31042
id=d11a6c0b-060d-4ce1-a652-6f648a363ba9
Oct 12 18:37:35 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-31042-11)
Oct 12 18:37:35 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [31042]
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting master-pgsql[pm1] = -INFINITY
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: master-pgsql=-INFINITY for pm1
Oct 12 18:37:35 [31042] pm1 crm_attribute: info: main: Update
master-pgsql=-INFINITY sent via attrd
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31042-14-header
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31042-14-header
Oct 12 18:37:35 [31042] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31042-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31042-14)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31042-14) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31042-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31042-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31042-14-header
Oct 12 18:37:35 [31042] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:35 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-31042-11)
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-31042-11) state:2
Oct 12 18:37:35 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-31042-11-header
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-31042-11-header
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-31042-11-header
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=31043
id=69a3d142-f802-4b9b-b528-db3c3261400b
Oct 12 18:37:35 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31043-14)
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31043]
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:35 [31043] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:35 [31043] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:37:35 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bab000 for uid=0 gid=0 pid=31043
id=96a392af-f106-46c0-96b4-d03d7ab84a00
Oct 12 18:37:35 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-31043-11)
Oct 12 18:37:35 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [31043]
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:35 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-master-baseline[pm1] = (null)
Oct 12 18:37:35 [11333] pm1 attrd: info: attrd_peer_update:
Setting pgsql-master-baseline[pm1]: 0000000000000000 -> (null) from pm1
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-master-baseline=(null) for pm1
Oct 12 18:37:35 [31043] pm1 crm_attribute: info: main: Update
pgsql-master-baseline=<none> sent via attrd
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:35 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31043-14)
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31043-14) state:2
Oct 12 18:37:35 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31043-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31043-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31043-14-header
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31043-14-header
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31043-14-header
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_process_modify:
Destroying
/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[6]
Oct 12 18:37:35 [11330] pm1 cib: debug: cib_process_modify:
Destroying
/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[6]
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.34 2
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.35 (null)
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: --
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-pgsql-master-baseline']
Oct 12 18:37:35 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=35
Oct 12 18:37:35 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.35 to 0.19.34
Oct 12 18:37:35 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.34 -> 0.19.35 (S_TRANSITION_ENGINE)
Oct 12 18:37:35 [11335] pm1 crmd: debug:
update_abort_priority: Abort priority upgraded from 0 to 1000000
Oct 12 18:37:35 [11335] pm1 crmd: debug:
update_abort_priority: Abort action done superseded by restart:
Transient attribute change
Oct 12 18:37:35 [11335] pm1 crmd: notice:
abort_transition_graph: Transition aborted by deletion of
nvpair[@id='status-1-pgsql-master-baseline']: Transient attribute change
(cib=0.19.35, source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-pgsql-master-baseline'],
0)
Oct 12 18:37:35 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=5, Pending=1, Fired=0, Skipped=0, Incomplete=27,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:35 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/attrd/20, version=0.19.35)
Oct 12 18:37:35 [31043] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31043-14-header
Oct 12 18:37:35 [31043] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:35 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-31043-11)
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-31043-11) state:2
Oct 12 18:37:35 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-31043-11-header
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-31043-11-header
Oct 12 18:37:35 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-31043-11-header
pgsql(pgsql)[30955]: 2015/10/12_18:37:35 INFO: Stopping PostgreSQL on
demote.
pgsql(pgsql)[30955]: 2015/10/12_18:37:36 INFO: waiting for server to
shut down.... done server stopped
pgsql(pgsql)[30955]: 2015/10/12_18:37:36 INFO: PostgreSQL is down
Oct 12 18:37:36 [31088] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:37:36 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=31088
id=01ca04d2-0ef1-4af5-a523-801e51345f0e
Oct 12 18:37:36 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31088-14)
Oct 12 18:37:36 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31088]
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31088] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31088] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31088] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31088] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:36 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:36 [31088] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:36 [31088] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:36 [31088] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:36 [31088] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:36 [31088] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:36 [31088] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:36 [31088] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:36 [31088] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:36 [31088] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:36 [31088] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:36 [31088] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:36 [31088] pm1 crm_mon: debug:
determine_op_status: pgsql_monitor_4000 on pm1 returned 'master' (8)
instead of the expected value: 'ok' (0)
Oct 12 18:37:36 [31088] pm1 crm_mon: warning:
unpack_rsc_op_failure: Processing failed op monitor for pgsql:1 on
pm1: master (8)
Oct 12 18:37:36 [31088] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:37:36 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31088-14)
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31088-14) state:2
Oct 12 18:37:36 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-31088-14-header
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-31088-14-header
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-31088-14-header
Oct 12 18:37:36 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=31090
id=1ca9204f-24fc-44ee-b1d2-4d2a75d67b71
Oct 12 18:37:36 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31090-14)
Oct 12 18:37:36 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31090]
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:36 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:36 [31090] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:36 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:36 [11330] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/status//node_state[@id='1']//transient_attributes//nvpair[@name='pgsql-status']
(/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3])
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
find_nvpair_attr_delegate: Match <nvpair id="status-1-pgsql-status"
name="pgsql-status" value="PRI"/>
Oct 12 18:37:36 [31090] pm1 crm_attribute: info: main: Read
pgsql-status=PRI
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31090-14-header
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31090-14-header
Oct 12 18:37:36 [31090] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31090-14-header
Oct 12 18:37:36 [31090] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:36 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31090-14)
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31090-14) state:2
Oct 12 18:37:36 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31090-14-header
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31090-14-header
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31090-14-header
pgsql(pgsql)[30955]: 2015/10/12_18:37:36 INFO: Changing pgsql-status
on pm1 : PRI->STOP.
Oct 12 18:37:36 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251b3d0 for uid=0 gid=0 pid=31095
id=46fe3471-15aa-469b-93b3-9d3255edf2a2
Oct 12 18:37:36 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31095-14)
Oct 12 18:37:36 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31095]
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:36 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:36 [31095] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:36 [31095] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:37:36 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bab000 for uid=0 gid=0 pid=31095
id=0fbe0815-629f-4458-8813-8ce990e0cc7f
Oct 12 18:37:36 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-31095-11)
Oct 12 18:37:36 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [31095]
Oct 12 18:37:36 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:36 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:36 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:36 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-status[pm1] = STOP
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-status=STOP for pm1
Oct 12 18:37:36 [31095] pm1 crm_attribute: info: main: Update
pgsql-status=STOP sent via attrd
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31095-14-header
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31095-14-header
Oct 12 18:37:36 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31095-14)
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31095-14) state:2
Oct 12 18:37:36 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31095-14-header
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31095-14-header
Oct 12 18:37:36 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31095-14-header
Oct 12 18:37:36 [11333] pm1 attrd: info: attrd_peer_update:
Setting pgsql-status[pm1]: PRI -> STOP from pm1
Oct 12 18:37:36 [31095] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31095-14-header
Oct 12 18:37:36 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:36 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.35 2
Oct 12 18:37:36 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.36 (null)
Oct 12 18:37:36 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=36
Oct 12 18:37:36 [11330] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-pgsql-status']:
@value=STOP
Oct 12 18:37:36 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.35 -> 0.19.36 (S_TRANSITION_ENGINE)
Oct 12 18:37:36 [11335] pm1 crmd: info:
abort_transition_graph: Transition aborted by status-1-pgsql-status,
pgsql-status=STOP: Transient attribute change (modify cib=0.19.36,
source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-pgsql-status'],
0)
Oct 12 18:37:36 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=5, Pending=1, Fired=0, Skipped=0, Incomplete=27,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:36 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/attrd/21, version=0.19.36)
Oct 12 18:37:36 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.36 to 0.19.35
Oct 12 18:37:36 [31095] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:36 [11332] pm1 lrmd: debug: operation_finished:
pgsql_demote_0:30955 - exited with rc=0
Oct 12 18:37:36 [11332] pm1 lrmd: debug: operation_finished:
pgsql_demote_0:30955:stderr [ -- empty -- ]
Oct 12 18:37:36 [11332] pm1 lrmd: debug: operation_finished:
pgsql_demote_0:30955:stdout [ -- empty -- ]
Oct 12 18:37:36 [11332] pm1 lrmd: info: log_finished: finished
- rsc:pgsql action:demote call_id:20 pid:30955 exit-code:0
exec-time:1229ms queue-time:0ms
Oct 12 18:37:36 [11335] pm1 crmd: debug:
create_operation_update: do_update_resource: Updating resource pgsql
after demote op complete (interval=0)
Oct 12 18:37:36 [11335] pm1 crmd: notice: process_lrm_event:
Operation pgsql_demote_0: ok (node=pm1, call=20, rc=0, cib-update=49,
confirmed=true)
Oct 12 18:37:36 [11335] pm1 crmd: debug:
update_history_cache: Updating history for 'pgsql' with demote op
Oct 12 18:37:36 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-31095-11)
Oct 12 18:37:36 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-31095-11) state:2
Oct 12 18:37:36 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:36 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-31095-11-header
Oct 12 18:37:36 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-31095-11-header
Oct 12 18:37:36 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-31095-11-header
Oct 12 18:37:36 [11330] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/49)
Oct 12 18:37:36 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:36 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.36 2
Oct 12 18:37:36 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.37 (null)
Oct 12 18:37:36 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=37
Oct 12 18:37:36 [11330] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='pgsql']/lrm_rsc_op[@id='pgsql_last_0']:
@operation_key=pgsql_demote_0, @operation=demote,
@transition-key=10:9:0:27561a4e-40a7-4815-98c0-df6b485550dc,
@transition-magic=0:0;10:9:0:27561a4e-40a7-4815-98c0-df6b485550dc,
@call-id=20, @last-run=1444664255, @last-rc-change=1444664255,
@exec-time=1229
Oct 12 18:37:36 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.37 to 0.19.36
Oct 12 18:37:36 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.36 -> 0.19.37 (S_TRANSITION_ENGINE)
Oct 12 18:37:36 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_demote_0 (10) confirmed on pm1 (rc=0)
Oct 12 18:37:36 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 31 fired and confirmed
Oct 12 18:37:36 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=6, Pending=0, Fired=1, Skipped=0, Incomplete=26,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:36 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 34 fired and confirmed
Oct 12 18:37:36 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=7, Pending=0, Fired=1, Skipped=0, Incomplete=25,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:36 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 52: notify pgsql_post_notify_demote_0 on pm2
Oct 12 18:37:36 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 54: notify pgsql_post_notify_demote_0 on pm1 (local)
Oct 12 18:37:36 [11335] pm1 crmd: info: do_lrm_rsc_op:
Performing key=54:9:0:27561a4e-40a7-4815-98c0-df6b485550dc op=pgsql_notify_0
Oct 12 18:37:36 [11332] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
a803ca7c-a9b1-490f-9ab6-5c18bb81be3f: rc=21, reply=1, notify=0, exit=4203648
Oct 12 18:37:36 [11332] pm1 lrmd: info: log_execute: executing
- rsc:pgsql action:notify call_id:21
Oct 12 18:37:36 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=8, Pending=2, Fired=2, Skipped=0, Incomplete=23,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:36 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/49, version=0.19.37)
Oct 12 18:37:37 [31130] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:37 [31130] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31130-24-header
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31130-24-header
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31130-24-header
Oct 12 18:37:37 [31130] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31130-24-header
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31130-24-header
Oct 12 18:37:37 [31130] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31130-24-header
Oct 12 18:37:37 [31130] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
pgsql(pgsql)[31096]: 2015/10/12_18:37:37 DEBUG: notify: post for demote
pgsql(pgsql)[31096]: 2015/10/12_18:37:37 DEBUG: post-demote called.
Demote uname is pm1
Oct 12 18:37:37 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31096 - exited with rc=0
Oct 12 18:37:37 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31096:stderr [ -- empty -- ]
Oct 12 18:37:37 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31096:stdout [ -- empty -- ]
Oct 12 18:37:37 [11332] pm1 lrmd: info: log_finished: finished
- rsc:pgsql action:notify call_id:21 pid:31096 exit-code:0
exec-time:101ms queue-time:0ms
Oct 12 18:37:37 [11335] pm1 crmd: debug:
create_operation_update: send_direct_ack: Updating resource pgsql
after notify op complete (interval=0)
Oct 12 18:37:37 [11335] pm1 crmd: debug: send_direct_ack:
ACK'ing resource op pgsql_notify_0 from
54:9:0:27561a4e-40a7-4815-98c0-df6b485550dc: lrm_invoke-lrmd-1444664257-44
Oct 12 18:37:37 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664257-44 from pm1
Oct 12 18:37:37 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (54) confirmed on pm1 (rc=0)
Oct 12 18:37:37 [11335] pm1 crmd: notice: process_lrm_event:
Operation pgsql_notify_0: ok (node=pm1, call=21, rc=0, cib-update=0,
confirmed=true)
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=9, Pending=1, Fired=0, Skipped=0, Incomplete=23,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:37 [11333] pm1 attrd: info: attrd_peer_update:
Setting pgsql-master-baseline[pm2]: (null) -> 0000000000000000 from pm2
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_process_modify:
Destroying
/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[9]
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.37 2
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.38 (null)
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=38
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']:
<nvpair id="status-2-pgsql-master-baseline" name="pgsql-master-baseline"
value="0000000000000000"/>
Oct 12 18:37:37 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/attrd/22, version=0.19.38)
Oct 12 18:37:37 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.38 to 0.19.37
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.37 -> 0.19.38 (S_TRANSITION_ENGINE)
Oct 12 18:37:37 [11335] pm1 crmd: info:
abort_transition_graph: Transition aborted by
status-2-pgsql-master-baseline, pgsql-master-baseline=0000000000000000:
Transient attribute change (create cib=0.19.38,
source=te_update_diff:391,
path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'],
0)
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=9, Pending=1, Fired=0, Skipped=0, Incomplete=23,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664257-9 from pm2
Oct 12 18:37:37 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (52) confirmed on pm2 (rc=0)
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 35 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 20 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
9 (Complete=10, Pending=0, Fired=2, Skipped=0, Incomplete=21,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: notice: run_graph: Transition
9 (Complete=12, Pending=0, Fired=0, Skipped=2, Incomplete=21,
Source=/var/lib/pacemaker/pengine/pe-input-103.bz2): Stopped
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_graph_trigger:
Transition 9 is now complete
Oct 12 18:37:37 [11335] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:37:37 [11335] pm1 crmd: debug: notify_crmd:
Transition 9 status: restart - Transient attribute change
Oct 12 18:37:37 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:37:37 [11335] pm1 crmd: info:
do_state_transition: State transition S_TRANSITION_ENGINE ->
S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:37:37 [11335] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:37:37 [11335] pm1 crmd: debug: do_pe_invoke: Query
50: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [11335] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=50,
ref=pe_calc-dc-1444664257-45, seq=164, quorate=1
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:37 [11334] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:37 [11334] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:37 [11334] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:37 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:37 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:37 [11334] pm1 pengine: debug:
determine_op_status: pgsql_monitor_4000 on pm1 returned 'master' (8)
instead of the expected value: 'ok' (0)
Oct 12 18:37:37 [11334] pm1 pengine: warning:
unpack_rsc_op_failure: Processing failed op monitor for pgsql:1 on
pm1: master (8)
Oct 12 18:37:37 [11334] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:37:37 [11334] pm1 pengine: info: native_print:
pgsql (ocf::heartbeat:pgsql): FAILED pm1
Oct 12 18:37:37 [11334] pm1 pengine: info: short_print:
Slaves: [ pm2 ]
Oct 12 18:37:37 [11334] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:37:37 [11334] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:37 [11334] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:37 [11334] pm1 pengine: info: get_failcount_full:
pgsql:0 has failed 1 times on pm1
Oct 12 18:37:37 [11334] pm1 pengine: info:
common_apply_stickiness: msPostgresql can fail 1 more times on pm1
before being forced off
Oct 12 18:37:37 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:1: preferring current
location (node=pm1, weight=1000000)
Oct 12 18:37:37 [11334] pm1 pengine: info: get_failcount_full:
pgsql:1 has failed 1 times on pm1
Oct 12 18:37:37 [11334] pm1 pengine: info:
common_apply_stickiness: msPostgresql can fail 1 more times on pm1
before being forced off
Oct 12 18:37:37 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:0: preferring current
location (node=pm2, weight=1000000)
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:0
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:1
Oct 12 18:37:37 [11334] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:37:37 [11334] pm1 pengine: debug: master_color: pgsql:0
master score: -1000000
Oct 12 18:37:37 [11334] pm1 pengine: debug: master_color: pgsql:1
master score: -1000000
Oct 12 18:37:37 [11334] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:37:37 [11334] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:37:37 [11334] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:37:37 [11334] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:37:37 [11334] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:37:37 [11334] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm1
Oct 12 18:37:37 [11334] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm1
Oct 12 18:37:37 [11334] pm1 pengine: info: LogActions: Leave
pgsql:0 (Slave pm2)
Oct 12 18:37:37 [11334] pm1 pengine: notice: LogActions: Recover
pgsql:1 (Slave pm1)
Oct 12 18:37:37 [11334] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:37:37 [11334] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: notice: process_pe_message:
Calculated Transition 10: /var/lib/pacemaker/pengine/pe-input-104.bz2
Oct 12 18:37:37 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:37 [11335] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:37 [11335] pm1 crmd: debug: unpack_graph: Unpacked
transition 10: 22 actions in 22 synapses
Oct 12 18:37:37 [11335] pm1 crmd: info: do_te_invoke:
Processing graph 10 (ref=pe_calc-dc-1444664257-45) derived from
/var/lib/pacemaker/pengine/pe-input-104.bz2
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 19 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
10 (Complete=0, Pending=0, Fired=1, Skipped=0, Incomplete=21,
Source=/var/lib/pacemaker/pengine/pe-input-104.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 47: notify pgsql_pre_notify_stop_0 on pm2
Oct 12 18:37:37 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 49: notify pgsql_pre_notify_stop_0 on pm1 (local)
Oct 12 18:37:37 [11335] pm1 crmd: info: do_lrm_rsc_op:
Performing key=49:10:0:27561a4e-40a7-4815-98c0-df6b485550dc
op=pgsql_notify_0
Oct 12 18:37:37 [11332] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
a803ca7c-a9b1-490f-9ab6-5c18bb81be3f: rc=22, reply=1, notify=0, exit=4203648
Oct 12 18:37:37 [11332] pm1 lrmd: info: log_execute: executing
- rsc:pgsql action:notify call_id:22
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
10 (Complete=1, Pending=2, Fired=2, Skipped=0, Incomplete=19,
Source=/var/lib/pacemaker/pengine/pe-input-104.bz2): In-progress
Oct 12 18:37:37 [31227] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:37 [31227] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31227-24-header
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31227-24-header
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31227-24-header
Oct 12 18:37:37 [31227] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31227-24-header
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31227-24-header
Oct 12 18:37:37 [31227] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31227-24-header
Oct 12 18:37:37 [31227] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
pgsql(pgsql)[31193]: 2015/10/12_18:37:37 DEBUG: notify: pre for stop
Oct 12 18:37:37 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31193 - exited with rc=0
Oct 12 18:37:37 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31193:stderr [ -- empty -- ]
Oct 12 18:37:37 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31193:stdout [ -- empty -- ]
Oct 12 18:37:37 [11332] pm1 lrmd: info: log_finished: finished
- rsc:pgsql action:notify call_id:22 pid:31193 exit-code:0
exec-time:95ms queue-time:0ms
Oct 12 18:37:37 [11335] pm1 crmd: debug:
create_operation_update: send_direct_ack: Updating resource pgsql
after notify op complete (interval=0)
Oct 12 18:37:37 [11335] pm1 crmd: debug: send_direct_ack:
ACK'ing resource op pgsql_notify_0 from
49:10:0:27561a4e-40a7-4815-98c0-df6b485550dc: lrm_invoke-lrmd-1444664257-48
Oct 12 18:37:37 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664257-48 from pm1
Oct 12 18:37:37 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (49) confirmed on pm1 (rc=0)
Oct 12 18:37:37 [11335] pm1 crmd: notice: process_lrm_event:
Operation pgsql_notify_0: ok (node=pm1, call=22, rc=0, cib-update=0,
confirmed=true)
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
10 (Complete=2, Pending=1, Fired=0, Skipped=0, Incomplete=19,
Source=/var/lib/pacemaker/pengine/pe-input-104.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664257-10 from pm2
Oct 12 18:37:37 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (47) confirmed on pm2 (rc=0)
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 20 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 17 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
10 (Complete=3, Pending=0, Fired=2, Skipped=0, Incomplete=17,
Source=/var/lib/pacemaker/pengine/pe-input-104.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 2: stop pgsql_stop_0 on pm1 (local)
Oct 12 18:37:37 [11335] pm1 crmd: debug: do_lrm_rsc_op: Stopped
0 recurring operations in preparation for pgsql_stop_0
Oct 12 18:37:37 [11335] pm1 crmd: info: do_lrm_rsc_op:
Performing key=2:10:0:27561a4e-40a7-4815-98c0-df6b485550dc op=pgsql_stop_0
Oct 12 18:37:37 [11332] pm1 lrmd: debug: create_lrmd_cmd:
Setting flag to leave pid group on timeout and only kill action pid for
pgsql_stop_0
Oct 12 18:37:37 [11332] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
a803ca7c-a9b1-490f-9ab6-5c18bb81be3f: rc=23, reply=1, notify=0, exit=4203648
Oct 12 18:37:37 [11332] pm1 lrmd: info: log_execute: executing
- rsc:pgsql action:stop call_id:23
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
10 (Complete=5, Pending=1, Fired=1, Skipped=0, Incomplete=16,
Source=/var/lib/pacemaker/pengine/pe-input-104.bz2): In-progress
Oct 12 18:37:37 [31315] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:37 [31315] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31315-24-header
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31315-24-header
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31315-24-header
Oct 12 18:37:37 [31315] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31315-24-header
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31315-24-header
Oct 12 18:37:37 [31315] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31315-24-header
Oct 12 18:37:37 [31315] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [31366] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:37 [31366] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31366-24-header
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31366-24-header
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31366-24-header
Oct 12 18:37:37 [31366] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31366-24-header
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31366-24-header
Oct 12 18:37:37 [31366] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31366-24-header
Oct 12 18:37:37 [31366] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31368
id=227b467f-c20e-4fb4-b476-7c4f67f72467
Oct 12 18:37:37 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31368-14)
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31368]
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:37 [31368] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:37 [31368] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:37:37 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bab000 for uid=0 gid=0 pid=31368
id=d0103284-78be-42ff-9377-4e7f285d6a69
Oct 12 18:37:37 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-31368-11)
Oct 12 18:37:37 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [31368]
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting master-pgsql[pm1] = -INFINITY
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: master-pgsql=-INFINITY for pm1
Oct 12 18:37:37 [31368] pm1 crm_attribute: info: main: Update
master-pgsql=-INFINITY sent via attrd
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31368-14-header
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31368-14-header
Oct 12 18:37:37 [31368] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31368-14-header
Oct 12 18:37:37 [31368] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31368-14)
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31368-14) state:2
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31368-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31368-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31368-14-header
Oct 12 18:37:37 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-31368-11)
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-31368-11) state:2
Oct 12 18:37:37 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-31368-11-header
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-31368-11-header
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-31368-11-header
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31369
id=c47e821c-162c-4ae2-913d-0cb590b6ed93
Oct 12 18:37:37 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31369-14)
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31369]
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:37 [31369] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:37 [31369] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:37:37 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bab000 for uid=0 gid=0 pid=31369
id=3fef9a2a-76e7-4107-99a3-eb8dffe568ab
Oct 12 18:37:37 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-31369-11)
Oct 12 18:37:37 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [31369]
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-xlog-loc[pm1] = (null)
Oct 12 18:37:37 [11333] pm1 attrd: info: attrd_peer_update:
Setting pgsql-xlog-loc[pm1]: 0000000000000000 -> (null) from pm1
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-xlog-loc=(null) for pm1
Oct 12 18:37:37 [31369] pm1 crm_attribute: info: main: Update
pgsql-xlog-loc=<none> sent via attrd
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31369-14)
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31369-14) state:2
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31369-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31369-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31369-14-header
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31369-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_process_modify:
Destroying
/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[5]
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.38 2
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.39 (null)
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: --
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-pgsql-xlog-loc']
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=39
Oct 12 18:37:37 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/attrd/23, version=0.19.39)
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.38 -> 0.19.39 (S_TRANSITION_ENGINE)
Oct 12 18:37:37 [11335] pm1 crmd: debug:
update_abort_priority: Abort priority upgraded from 0 to 1000000
Oct 12 18:37:37 [11335] pm1 crmd: debug:
update_abort_priority: Abort action done superseded by restart:
Transient attribute change
Oct 12 18:37:37 [11335] pm1 crmd: notice:
abort_transition_graph: Transition aborted by deletion of
nvpair[@id='status-1-pgsql-xlog-loc']: Transient attribute change
(cib=0.19.39, source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-pgsql-xlog-loc'],
0)
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
10 (Complete=5, Pending=1, Fired=0, Skipped=0, Incomplete=16,
Source=/var/lib/pacemaker/pengine/pe-input-104.bz2): In-progress
Oct 12 18:37:37 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.39 to 0.19.38
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31369-14-header
Oct 12 18:37:37 [31369] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31369-14-header
Oct 12 18:37:37 [31369] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-31369-11)
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-31369-11) state:2
Oct 12 18:37:37 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-31369-11-header
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-31369-11-header
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-31369-11-header
pgsql(pgsql)[31281]: 2015/10/12_18:37:37 INFO: PostgreSQL is already
stopped.
Oct 12 18:37:37 [31380] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31380
id=114703ee-f3e9-41d2-89bc-82d74713b4a1
Oct 12 18:37:37 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31380-14)
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31380]
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31380] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31380] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31380] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31380] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [31380] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:37 [31380] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:37 [31380] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:37 [31380] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:37 [31380] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:37 [31380] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:37 [31380] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:37 [31380] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:37 [31380] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:37 [31380] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:37 [31380] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:37 [31380] pm1 crm_mon: debug:
determine_op_status: pgsql_monitor_4000 on pm1 returned 'master' (8)
instead of the expected value: 'ok' (0)
Oct 12 18:37:37 [31380] pm1 crm_mon: warning:
unpack_rsc_op_failure: Processing failed op monitor for pgsql:1 on
pm1: master (8)
Oct 12 18:37:37 [31380] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31380-14)
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31380-14) state:2
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-31380-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-31380-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-31380-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31382
id=421eb65f-99d1-4644-bdb0-d8b465e3a869
Oct 12 18:37:37 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31382-14)
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31382]
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:37 [31382] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/status//node_state[@id='1']//transient_attributes//nvpair[@name='pgsql-status']
(/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3])
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
find_nvpair_attr_delegate: Match <nvpair id="status-1-pgsql-status"
name="pgsql-status" value="STOP"/>
Oct 12 18:37:37 [31382] pm1 crm_attribute: info: main: Read
pgsql-status=STOP
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31382-14-header
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31382-14-header
Oct 12 18:37:37 [31382] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31382-14-header
Oct 12 18:37:37 [31382] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [11332] pm1 lrmd: debug: operation_finished:
pgsql_stop_0:31281 - exited with rc=0
Oct 12 18:37:37 [11332] pm1 lrmd: debug: operation_finished:
pgsql_stop_0:31281:stderr [ -- empty -- ]
Oct 12 18:37:37 [11332] pm1 lrmd: debug: operation_finished:
pgsql_stop_0:31281:stdout [ -- empty -- ]
Oct 12 18:37:37 [11332] pm1 lrmd: info: log_finished: finished
- rsc:pgsql action:stop call_id:23 pid:31281 exit-code:0 exec-time:168ms
queue-time:0ms
Oct 12 18:37:37 [11335] pm1 crmd: debug:
create_operation_update: do_update_resource: Updating resource pgsql
after stop op complete (interval=0)
Oct 12 18:37:37 [11335] pm1 crmd: notice: process_lrm_event:
Operation pgsql_stop_0: ok (node=pm1, call=23, rc=0, cib-update=51,
confirmed=true)
Oct 12 18:37:37 [11335] pm1 crmd: debug:
update_history_cache: Updating history for 'pgsql' with stop op
Oct 12 18:37:37 [11330] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/51)
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31382-14)
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31382-14) state:2
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31382-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31382-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31382-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.39 2
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.40 (null)
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=40
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='pgsql']/lrm_rsc_op[@id='pgsql_last_0']:
@operation_key=pgsql_stop_0, @operation=stop,
@transition-key=2:10:0:27561a4e-40a7-4815-98c0-df6b485550dc,
@transition-magic=0:0;2:10:0:27561a4e-40a7-4815-98c0-df6b485550dc,
@call-id=23, @last-run=1444664257, @last-rc-change=1444664257,
@exec-time=168
Oct 12 18:37:37 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/51, version=0.19.40)
Oct 12 18:37:37 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.40 to 0.19.39
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.39 -> 0.19.40 (S_TRANSITION_ENGINE)
Oct 12 18:37:37 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_stop_0 (2) confirmed on pm1 (rc=0)
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 18 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
10 (Complete=6, Pending=0, Fired=1, Skipped=0, Incomplete=15,
Source=/var/lib/pacemaker/pengine/pe-input-104.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 21 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
10 (Complete=7, Pending=0, Fired=1, Skipped=0, Incomplete=14,
Source=/var/lib/pacemaker/pengine/pe-input-104.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 48: notify pgsql_post_notify_stop_0 on pm2
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
10 (Complete=8, Pending=1, Fired=1, Skipped=0, Incomplete=13,
Source=/var/lib/pacemaker/pengine/pe-input-104.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664257-11 from pm2
Oct 12 18:37:37 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (48) confirmed on pm2 (rc=0)
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 22 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 13 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 3 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
10 (Complete=9, Pending=0, Fired=3, Skipped=0, Incomplete=10,
Source=/var/lib/pacemaker/pengine/pe-input-104.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: notice: run_graph: Transition
10 (Complete=12, Pending=0, Fired=0, Skipped=1, Incomplete=10,
Source=/var/lib/pacemaker/pengine/pe-input-104.bz2): Stopped
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_graph_trigger:
Transition 10 is now complete
Oct 12 18:37:37 [11335] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:37:37 [11335] pm1 crmd: debug: notify_crmd:
Transition 10 status: restart - Transient attribute change
Oct 12 18:37:37 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:37:37 [11335] pm1 crmd: info:
do_state_transition: State transition S_TRANSITION_ENGINE ->
S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:37:37 [11335] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:37:37 [11335] pm1 crmd: debug: do_pe_invoke: Query
52: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [11335] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=52,
ref=pe_calc-dc-1444664257-51, seq=164, quorate=1
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:37 [11334] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:37 [11334] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:37 [11334] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:37 [11334] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:37 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:37 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:37 [11334] pm1 pengine: debug:
determine_op_status: pgsql_monitor_4000 on pm1 returned 'master' (8)
instead of the expected value: 'ok' (0)
Oct 12 18:37:37 [11334] pm1 pengine: warning:
unpack_rsc_op_failure: Processing failed op monitor for pgsql:1 on
pm1: master (8)
Oct 12 18:37:37 [11334] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:37 [11334] pm1 pengine: info: short_print:
Slaves: [ pm2 ]
Oct 12 18:37:37 [11334] pm1 pengine: info: short_print:
Stopped: [ pm1 ]
Oct 12 18:37:37 [11334] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:37:37 [11334] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:37 [11334] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:37 [11334] pm1 pengine: info: get_failcount_full:
pgsql:0 has failed 1 times on pm1
Oct 12 18:37:37 [11334] pm1 pengine: info:
common_apply_stickiness: msPostgresql can fail 1 more times on pm1
before being forced off
Oct 12 18:37:37 [11334] pm1 pengine: info: get_failcount_full:
pgsql:1 has failed 1 times on pm1
Oct 12 18:37:37 [11334] pm1 pengine: info:
common_apply_stickiness: msPostgresql can fail 1 more times on pm1
before being forced off
Oct 12 18:37:37 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:0: preferring current
location (node=pm2, weight=1000000)
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:0
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:1
Oct 12 18:37:37 [11334] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:37:37 [11334] pm1 pengine: debug: master_color: pgsql:0
master score: -1000000
Oct 12 18:37:37 [11334] pm1 pengine: debug: master_color: pgsql:1
master score: -1000000
Oct 12 18:37:37 [11334] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:37:37 [11334] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:37:37 [11334] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:37:37 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:37:37 [11334] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:37:37 [11334] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:37:37 [11334] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm1
Oct 12 18:37:37 [11334] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm1
Oct 12 18:37:37 [11334] pm1 pengine: info: LogActions: Leave
pgsql:0 (Slave pm2)
Oct 12 18:37:37 [11334] pm1 pengine: notice: LogActions: Start
pgsql:1 (pm1)
Oct 12 18:37:37 [11334] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:37:37 [11334] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:37 [11334] pm1 pengine: notice: process_pe_message:
Calculated Transition 11: /var/lib/pacemaker/pengine/pe-input-105.bz2
Oct 12 18:37:37 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:37 [11335] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:37 [11335] pm1 crmd: debug: unpack_graph: Unpacked
transition 11: 11 actions in 11 synapses
Oct 12 18:37:37 [11335] pm1 crmd: info: do_te_invoke:
Processing graph 11 (ref=pe_calc-dc-1444664257-51) derived from
/var/lib/pacemaker/pengine/pe-input-105.bz2
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 12 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
11 (Complete=0, Pending=0, Fired=1, Skipped=0, Incomplete=10,
Source=/var/lib/pacemaker/pengine/pe-input-105.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 44: notify pgsql_pre_notify_start_0 on pm2
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
11 (Complete=1, Pending=1, Fired=1, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-input-105.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664257-12 from pm2
Oct 12 18:37:37 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (44) confirmed on pm2 (rc=0)
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 13 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 10 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
11 (Complete=2, Pending=0, Fired=2, Skipped=0, Incomplete=7,
Source=/var/lib/pacemaker/pengine/pe-input-105.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 8: start pgsql_start_0 on pm1 (local)
Oct 12 18:37:37 [11335] pm1 crmd: debug: do_lrm_rsc_op: Stopped
0 recurring operations in preparation for pgsql_start_0
Oct 12 18:37:37 [11335] pm1 crmd: info: do_lrm_rsc_op:
Performing key=8:11:0:27561a4e-40a7-4815-98c0-df6b485550dc op=pgsql_start_0
Oct 12 18:37:37 [11332] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
a803ca7c-a9b1-490f-9ab6-5c18bb81be3f: rc=24, reply=1, notify=0, exit=4203648
Oct 12 18:37:37 [11332] pm1 lrmd: info: log_execute: executing
- rsc:pgsql action:start call_id:24
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
11 (Complete=4, Pending=1, Fired=1, Skipped=0, Incomplete=6,
Source=/var/lib/pacemaker/pengine/pe-input-105.bz2): In-progress
Oct 12 18:37:37 [31417] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:37 [31417] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31417-24-header
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31417-24-header
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31417-24-header
Oct 12 18:37:37 [31417] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31417-24-header
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31417-24-header
Oct 12 18:37:37 [31417] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31417-24-header
Oct 12 18:37:37 [31417] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [31473] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31473
id=d412a70b-6018-4935-b173-1141e20f3af8
Oct 12 18:37:37 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31473-14)
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31473]
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31473] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31473] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31473] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31473] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [31473] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:37 [31473] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:37 [31473] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:37 [31473] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:37 [31473] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:37 [31473] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:37 [31473] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:37 [31473] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:37 [31473] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:37 [31473] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:37 [31473] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:37 [31473] pm1 crm_mon: debug:
determine_op_status: pgsql_monitor_4000 on pm1 returned 'master' (8)
instead of the expected value: 'ok' (0)
Oct 12 18:37:37 [31473] pm1 crm_mon: warning:
unpack_rsc_op_failure: Processing failed op monitor for pgsql:1 on
pm1: master (8)
Oct 12 18:37:37 [31473] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31473-14)
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31473-14) state:2
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-31473-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-31473-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-31473-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31475
id=4b00349c-ed8a-4bff-8f85-e50fe08c1302
Oct 12 18:37:37 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31475-14)
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31475]
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:37 [31475] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/status//node_state[@id='1']//transient_attributes//nvpair[@name='pgsql-status']
(/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3])
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
find_nvpair_attr_delegate: Match <nvpair id="status-1-pgsql-status"
name="pgsql-status" value="STOP"/>
Oct 12 18:37:37 [31475] pm1 crm_attribute: info: main: Read
pgsql-status=STOP
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31475-14-header
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31475-14-header
Oct 12 18:37:37 [31475] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31475-14-header
Oct 12 18:37:37 [31475] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31475-14)
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31475-14) state:2
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31475-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31475-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31475-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31476
id=4558a999-a1b7-4152-bc1e-6a13afbbd0ad
Oct 12 18:37:37 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31476-14)
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31476]
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:37 [31476] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:37 [31476] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:37:37 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bab000 for uid=0 gid=0 pid=31476
id=1a04e4fd-788c-453f-a189-ae0008df0346
Oct 12 18:37:37 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-31476-11)
Oct 12 18:37:37 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [31476]
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-master-baseline[pm1] = (null)
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-master-baseline=(null) for pm1
Oct 12 18:37:37 [31476] pm1 crm_attribute: info: main: Update
pgsql-master-baseline=<none> sent via attrd
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31476-14-header
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31476-14-header
Oct 12 18:37:37 [31476] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31476-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31476-14)
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31476-14) state:2
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31476-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31476-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31476-14-header
Oct 12 18:37:37 [31476] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-31476-11)
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-31476-11) state:2
Oct 12 18:37:37 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-31476-11-header
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-31476-11-header
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-31476-11-header
Oct 12 18:37:37 [31478] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:37 [31478] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31478-24-header
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31478-24-header
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31478-24-header
Oct 12 18:37:37 [31478] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31478-24-header
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31478-24-header
Oct 12 18:37:37 [31478] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31478-24-header
Oct 12 18:37:37 [31478] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31480
id=28143b56-9bd8-4a19-a4c1-4e79c8750635
Oct 12 18:37:37 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31480-14)
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31480]
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:37 [31480] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:37 [31480] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:37:37 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bab000 for uid=0 gid=0 pid=31480
id=d5bf5f0a-4405-477d-8be0-347011779857
Oct 12 18:37:37 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-31480-11)
Oct 12 18:37:37 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [31480]
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting master-pgsql[pm1] = -INFINITY
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: master-pgsql=-INFINITY for pm1
Oct 12 18:37:37 [31480] pm1 crm_attribute: info: main: Update
master-pgsql=-INFINITY sent via attrd
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31480-14-header
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31480-14-header
Oct 12 18:37:37 [31480] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31480-14-header
Oct 12 18:37:37 [31480] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31480-14)
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31480-14) state:2
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31480-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31480-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31480-14-header
Oct 12 18:37:37 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-31480-11)
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-31480-11) state:2
Oct 12 18:37:37 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-31480-11-header
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-31480-11-header
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-31480-11-header
pgsql(pgsql)[31383]: 2015/10/12_18:37:37 DEBUG: Created
recovery.conf. host=10.200.200.225, user=postgres
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31495
id=92ea7fe0-c11c-4ecb-8bb7-f1ef38d192bc
Oct 12 18:37:37 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31495-14)
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31495]
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:37 [31495] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:37 [31495] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:37:37 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bab000 for uid=0 gid=0 pid=31495
id=373b4624-c0b3-4261-9a44-2fa2ca9c014a
Oct 12 18:37:37 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-31495-11)
Oct 12 18:37:37 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [31495]
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:37 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-xlog-loc[pm1] = (null)
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-xlog-loc=(null) for pm1
Oct 12 18:37:37 [31495] pm1 crm_attribute: info: main: Update
pgsql-xlog-loc=<none> sent via attrd
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31495-14-header
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31495-14-header
Oct 12 18:37:37 [31495] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31495-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31495-14)
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31495-14) state:2
Oct 12 18:37:37 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31495-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31495-14-header
Oct 12 18:37:37 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31495-14-header
Oct 12 18:37:37 [31495] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-31495-11)
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-31495-11) state:2
Oct 12 18:37:37 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-31495-11-header
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-31495-11-header
Oct 12 18:37:37 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-31495-11-header
pgsql(pgsql)[31383]: 2015/10/12_18:37:37 INFO: Set all nodes into
async mode.
pgsql(pgsql)[31383]: 2015/10/12_18:37:37 ERROR: My data may be
inconsistent. You have to remove /var/lib/pgsql/9.3/tmp//PGSQL.lock file
to force start.
Oct 12 18:37:37 [11332] pm1 lrmd: debug: operation_finished:
pgsql_start_0:31383 - exited with rc=1
Oct 12 18:37:37 [11332] pm1 lrmd: notice: operation_finished:
pgsql_start_0:31383:stderr [ ocf-exit-reason:My data may be
inconsistent. You have to remove /var/lib/pgsql/9.3/tmp//PGSQL.lock file
to force start. ]
Oct 12 18:37:37 [11332] pm1 lrmd: debug: operation_finished:
pgsql_start_0:31383:stdout [ -- empty -- ]
Oct 12 18:37:37 [11332] pm1 lrmd: info: log_finished: finished
- rsc:pgsql action:start call_id:24 pid:31383 exit-code:1
exec-time:209ms queue-time:0ms
Oct 12 18:37:37 [11335] pm1 crmd: debug:
create_operation_update: do_update_resource: Updating resource pgsql
after start op complete (interval=0)
Oct 12 18:37:37 [31546] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:37 [31546] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31546-24-header
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31546-24-header
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31546-24-header
Oct 12 18:37:37 [31546] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31546-24-header
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31546-24-header
Oct 12 18:37:37 [31546] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31546-24-header
Oct 12 18:37:37 [31546] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:37 [11335] pm1 crmd: info: action_synced_wait:
Managed pgsql_meta-data_0 process 31512 exited with rc=0
Oct 12 18:37:37 [11335] pm1 crmd: notice: process_lrm_event:
Operation pgsql_start_0: unknown error (node=pm1, call=24, rc=1,
cib-update=53, confirmed=true)
Oct 12 18:37:37 [11335] pm1 crmd: notice: process_lrm_event:
pm1-pgsql_start_0:24 [ ocf-exit-reason:My data may be inconsistent. You
have to remove /var/lib/pgsql/9.3/tmp//PGSQL.lock file to force start.\n ]
Oct 12 18:37:37 [11335] pm1 crmd: debug:
update_history_cache: Updating history for 'pgsql' with start op
Oct 12 18:37:37 [11330] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/53)
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.40 2
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.41 (null)
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=41
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='pgsql']/lrm_rsc_op[@id='pgsql_last_0']:
@operation_key=pgsql_start_0, @operation=start,
@transition-key=8:11:0:27561a4e-40a7-4815-98c0-df6b485550dc,
@transition-magic=0:1;8:11:0:27561a4e-40a7-4815-98c0-df6b485550dc,
@call-id=24, @rc-code=1, @exec-time=209, @exit-reason=My data may be
inconsistent. You have to remove /var/lib/pgsql/9.3/tmp//PGSQL.lock file
to for
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='pgsql']/lrm_rsc_op[@id='pgsql_last_failure_0']:
@operation_key=pgsql_start_0, @operation=start,
@transition-key=8:11:0:27561a4e-40a7-4815-98c0-df6b485550dc,
@transition-magic=0:1;8:11:0:27561a4e-40a7-4815-98c0-df6b485550dc,
@call-id=24, @rc-code=1, @interval=0, @last-rc-change=1444664257,
@exec-time=209, @op-digest=7d959c3527b20b7148f043317fcacf4d, @exit-reason=
Oct 12 18:37:37 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/53, version=0.19.41)
Oct 12 18:37:37 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.41 to 0.19.40
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.40 -> 0.19.41 (S_TRANSITION_ENGINE)
Oct 12 18:37:37 [11335] pm1 crmd: warning: status_from_rc: Action
8 (pgsql_start_0) on pm1 failed (target: 0 vs. rc: 1): Error
Oct 12 18:37:37 [11335] pm1 crmd: debug:
update_abort_priority: Abort priority upgraded from 0 to 1
Oct 12 18:37:37 [11335] pm1 crmd: debug:
update_abort_priority: Abort action done superseded by restart: Event
failed
Oct 12 18:37:37 [11335] pm1 crmd: notice:
abort_transition_graph: Transition aborted by pgsql_start_0 'modify'
on pm1: Event failed
(magic=0:1;8:11:0:27561a4e-40a7-4815-98c0-df6b485550dc, cib=0.19.41,
source=match_graph_event:381, 0)
Oct 12 18:37:37 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_start_0 (8) confirmed on pm1 (rc=1)
Oct 12 18:37:37 [11335] pm1 crmd: info: update_failcount:
Updating failcount for pgsql on pm1 after failed start: rc=1
(update=INFINITY, time=1444664257)
Oct 12 18:37:37 [11335] pm1 crmd: debug:
attrd_update_delegate: Sent update: fail-count-pgsql=INFINITY for pm1
Oct 12 18:37:37 [11335] pm1 crmd: debug:
attrd_update_delegate: Sent update: last-failure-pgsql=1444664257 for pm1
Oct 12 18:37:37 [11335] pm1 crmd: info:
process_graph_event: Detected action (11.8) pgsql_start_0.24=unknown
error: failed
Oct 12 18:37:37 [11335] pm1 crmd: warning: status_from_rc: Action
8 (pgsql_start_0) on pm1 failed (target: 0 vs. rc: 1): Error
Oct 12 18:37:37 [11335] pm1 crmd: info:
abort_transition_graph: Transition aborted by pgsql_start_0 'modify'
on pm1: Event failed
(magic=0:1;8:11:0:27561a4e-40a7-4815-98c0-df6b485550dc, cib=0.19.41,
source=match_graph_event:381, 0)
Oct 12 18:37:37 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_start_0 (8) confirmed on pm1 (rc=1)
Oct 12 18:37:37 [11335] pm1 crmd: info: update_failcount:
Updating failcount for pgsql on pm1 after failed start: rc=1
(update=INFINITY, time=1444664257)
Oct 12 18:37:37 [11335] pm1 crmd: debug:
attrd_update_delegate: Sent update: fail-count-pgsql=INFINITY for pm1
Oct 12 18:37:37 [11335] pm1 crmd: debug:
attrd_update_delegate: Sent update: last-failure-pgsql=1444664257 for pm1
Oct 12 18:37:37 [11335] pm1 crmd: info:
process_graph_event: Detected action (11.8) pgsql_start_0.24=unknown
error: failed
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 11 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
11 (Complete=5, Pending=0, Fired=1, Skipped=0, Incomplete=5,
Source=/var/lib/pacemaker/pengine/pe-input-105.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 14 fired and confirmed
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
11 (Complete=6, Pending=0, Fired=1, Skipped=0, Incomplete=4,
Source=/var/lib/pacemaker/pengine/pe-input-105.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 45: notify pgsql_post_notify_start_0 on pm2
Oct 12 18:37:37 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting fail-count-pgsql[pm1] = INFINITY
Oct 12 18:37:37 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting last-failure-pgsql[pm1] = 1444664257
Oct 12 18:37:37 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting fail-count-pgsql[pm1] = INFINITY
Oct 12 18:37:37 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting last-failure-pgsql[pm1] = 1444664257
Oct 12 18:37:37 [11333] pm1 attrd: info: attrd_peer_update:
Setting fail-count-pgsql[pm1]: 1 -> INFINITY from pm1
Oct 12 18:37:37 [11333] pm1 attrd: info: attrd_peer_update:
Setting last-failure-pgsql[pm1]: 1444664255 -> 1444664257 from pm1
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.41 2
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.42 (null)
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=42
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-pgsql']:
@value=INFINITY
Oct 12 18:37:37 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/attrd/24, version=0.19.42)
Oct 12 18:37:37 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.42 to 0.19.41
Oct 12 18:37:37 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.42 2
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.43 (null)
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=43
Oct 12 18:37:37 [11330] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-pgsql']:
@value=1444664257
Oct 12 18:37:37 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/attrd/25, version=0.19.43)
Oct 12 18:37:37 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.43 to 0.19.42
Oct 12 18:37:37 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 46: notify pgsql_post_notify_start_0 on pm1 (local)
Oct 12 18:37:37 [11335] pm1 crmd: info: do_lrm_rsc_op:
Performing key=46:11:0:27561a4e-40a7-4815-98c0-df6b485550dc
op=pgsql_notify_0
Oct 12 18:37:37 [11332] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
a803ca7c-a9b1-490f-9ab6-5c18bb81be3f: rc=25, reply=1, notify=0, exit=4203648
Oct 12 18:37:37 [11332] pm1 lrmd: info: log_execute: executing
- rsc:pgsql action:notify call_id:25
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
11 (Complete=7, Pending=2, Fired=2, Skipped=0, Incomplete=2,
Source=/var/lib/pacemaker/pengine/pe-input-105.bz2): In-progress
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.41 -> 0.19.42 (S_TRANSITION_ENGINE)
Oct 12 18:37:37 [11335] pm1 crmd: debug:
update_abort_priority: Abort priority upgraded from 1 to 1000000
Oct 12 18:37:37 [11335] pm1 crmd: debug:
update_abort_priority: 'Event failed' abort superseded by Transient
attribute change
Oct 12 18:37:37 [11335] pm1 crmd: notice:
abort_transition_graph: Transition aborted by
status-1-fail-count-pgsql, fail-count-pgsql=INFINITY: Transient
attribute change (modify cib=0.19.42, source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-fail-count-pgsql'],
0)
Oct 12 18:37:37 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.42 -> 0.19.43 (S_TRANSITION_ENGINE)
Oct 12 18:37:37 [11335] pm1 crmd: info:
abort_transition_graph: Transition aborted by
status-1-last-failure-pgsql, last-failure-pgsql=1444664257: Transient
attribute change (modify cib=0.19.43, source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-last-failure-pgsql'],
0)
Oct 12 18:37:37 [11335] pm1 crmd: debug: run_graph: Transition
11 (Complete=7, Pending=2, Fired=0, Skipped=0, Incomplete=2,
Source=/var/lib/pacemaker/pengine/pe-input-105.bz2): In-progress
Oct 12 18:37:38 [31582] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:38 [31582] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31582-24-header
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31582-24-header
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31582-24-header
Oct 12 18:37:38 [31582] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31582-24-header
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31582-24-header
Oct 12 18:37:38 [31582] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31582-24-header
Oct 12 18:37:38 [31582] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
pgsql(pgsql)[31548]: 2015/10/12_18:37:38 DEBUG: notify: post for start
Oct 12 18:37:38 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31548 - exited with rc=0
Oct 12 18:37:38 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31548:stderr [ -- empty -- ]
Oct 12 18:37:38 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31548:stdout [ -- empty -- ]
Oct 12 18:37:38 [11332] pm1 lrmd: info: log_finished: finished
- rsc:pgsql action:notify call_id:25 pid:31548 exit-code:0
exec-time:91ms queue-time:0ms
Oct 12 18:37:38 [11335] pm1 crmd: debug:
create_operation_update: send_direct_ack: Updating resource pgsql
after notify op complete (interval=0)
Oct 12 18:37:38 [11335] pm1 crmd: debug: send_direct_ack:
ACK'ing resource op pgsql_notify_0 from
46:11:0:27561a4e-40a7-4815-98c0-df6b485550dc: lrm_invoke-lrmd-1444664258-56
Oct 12 18:37:38 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664258-56 from pm1
Oct 12 18:37:38 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (46) confirmed on pm1 (rc=0)
Oct 12 18:37:38 [11335] pm1 crmd: notice: process_lrm_event:
Operation pgsql_notify_0: ok (node=pm1, call=25, rc=0, cib-update=0,
confirmed=true)
Oct 12 18:37:38 [11335] pm1 crmd: debug: run_graph: Transition
11 (Complete=8, Pending=1, Fired=0, Skipped=0, Incomplete=2,
Source=/var/lib/pacemaker/pengine/pe-input-105.bz2): In-progress
Oct 12 18:37:38 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664258-13 from pm2
Oct 12 18:37:38 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (45) confirmed on pm2 (rc=0)
Oct 12 18:37:38 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 15 fired and confirmed
Oct 12 18:37:38 [11335] pm1 crmd: debug: run_graph: Transition
11 (Complete=9, Pending=0, Fired=1, Skipped=0, Incomplete=1,
Source=/var/lib/pacemaker/pengine/pe-input-105.bz2): In-progress
Oct 12 18:37:38 [11335] pm1 crmd: notice: run_graph: Transition
11 (Complete=10, Pending=0, Fired=0, Skipped=0, Incomplete=1,
Source=/var/lib/pacemaker/pengine/pe-input-105.bz2): Complete
Oct 12 18:37:38 [11335] pm1 crmd: debug: te_graph_trigger:
Transition 11 is now complete
Oct 12 18:37:38 [11335] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:37:38 [11335] pm1 crmd: debug: notify_crmd:
Transition 11 status: restart - Transient attribute change
Oct 12 18:37:38 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:37:38 [11335] pm1 crmd: info:
do_state_transition: State transition S_TRANSITION_ENGINE ->
S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:37:38 [11335] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:37:38 [11335] pm1 crmd: debug: do_pe_invoke: Query
54: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:37:38 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:38 [11335] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=54,
ref=pe_calc-dc-1444664258-57, seq=164, quorate=1
Oct 12 18:37:38 [11334] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:38 [11334] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:38 [11334] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:38 [11334] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:38 [11334] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:38 [11334] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:38 [11334] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:38 [11334] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:38 [11334] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:38 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:38 [11334] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:38 [11334] pm1 pengine: debug:
determine_op_status: pgsql_start_0 on pm1 returned 'unknown error'
(1) instead of the expected value: 'ok' (0)
Oct 12 18:37:38 [11334] pm1 pengine: warning:
unpack_rsc_op_failure: Processing failed op start for pgsql:1 on pm1:
unknown error (1)
Oct 12 18:37:38 [11334] pm1 pengine: debug:
determine_op_status: pgsql_start_0 on pm1 returned 'unknown error'
(1) instead of the expected value: 'ok' (0)
Oct 12 18:37:38 [11334] pm1 pengine: warning:
unpack_rsc_op_failure: Processing failed op start for pgsql:1 on pm1:
unknown error (1)
Oct 12 18:37:38 [11334] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:37:38 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:38 [11334] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:37:38 [11334] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:37:38 [11334] pm1 pengine: info: native_print:
pgsql (ocf::heartbeat:pgsql): FAILED pm1
Oct 12 18:37:38 [11334] pm1 pengine: info: short_print:
Slaves: [ pm2 ]
Oct 12 18:37:38 [11334] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:37:38 [11334] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:38 [11334] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:37:38 [11334] pm1 pengine: info: get_failcount_full:
pgsql:0 has failed INFINITY times on pm1
Oct 12 18:37:38 [11334] pm1 pengine: warning:
common_apply_stickiness: Forcing msPostgresql away from pm1 after
1000000 failures (max=2)
Oct 12 18:37:38 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:1: preferring current
location (node=pm1, weight=1000000)
Oct 12 18:37:38 [11334] pm1 pengine: info: get_failcount_full:
pgsql:1 has failed INFINITY times on pm1
Oct 12 18:37:38 [11334] pm1 pengine: warning:
common_apply_stickiness: Forcing msPostgresql away from pm1 after
1000000 failures (max=2)
Oct 12 18:37:38 [11334] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:0: preferring current
location (node=pm2, weight=1000000)
Oct 12 18:37:38 [11334] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:0
Oct 12 18:37:38 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource pgsql:1 are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:37:38 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for pgsql:1
Oct 12 18:37:38 [11334] pm1 pengine: debug: native_assign_node:
Processing pgsql:1_stop_0
Oct 12 18:37:38 [11334] pm1 pengine: info: native_color: Resource
pgsql:1 cannot run anywhere
Oct 12 18:37:38 [11334] pm1 pengine: debug: clone_color: Allocated
1 msPostgresql instances of a possible 2
Oct 12 18:37:38 [11334] pm1 pengine: debug: master_color: pgsql:1
master score: 0
Oct 12 18:37:38 [11334] pm1 pengine: debug: master_color: pgsql:0
master score: -1000000
Oct 12 18:37:38 [11334] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:37:38 [11334] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:37:38 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:37:38 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:37:38 [11334] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:37:38 [11334] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:37:38 [11334] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:37:38 [11334] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:37:38 [11334] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:37:38 [11334] pm1 pengine: info: LogActions: Leave
pgsql:0 (Slave pm2)
Oct 12 18:37:38 [11334] pm1 pengine: notice: LogActions: Stop
pgsql:1 (pm1)
Oct 12 18:37:38 [11334] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:37:38 [11334] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:37:38 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:38 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:37:38 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:38 [11334] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:37:38 [11334] pm1 pengine: notice: process_pe_message:
Calculated Transition 12: /var/lib/pacemaker/pengine/pe-input-106.bz2
Oct 12 18:37:38 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:38 [11335] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:37:38 [11335] pm1 crmd: debug: unpack_graph: Unpacked
transition 12: 11 actions in 11 synapses
Oct 12 18:37:38 [11335] pm1 crmd: info: do_te_invoke:
Processing graph 12 (ref=pe_calc-dc-1444664258-57) derived from
/var/lib/pacemaker/pengine/pe-input-106.bz2
Oct 12 18:37:38 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 17 fired and confirmed
Oct 12 18:37:38 [11335] pm1 crmd: debug: run_graph: Transition
12 (Complete=0, Pending=0, Fired=1, Skipped=0, Incomplete=10,
Source=/var/lib/pacemaker/pengine/pe-input-106.bz2): In-progress
Oct 12 18:37:38 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 45: notify pgsql_pre_notify_stop_0 on pm2
Oct 12 18:37:38 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 47: notify pgsql_pre_notify_stop_0 on pm1 (local)
Oct 12 18:37:38 [11335] pm1 crmd: info: do_lrm_rsc_op:
Performing key=47:12:0:27561a4e-40a7-4815-98c0-df6b485550dc
op=pgsql_notify_0
Oct 12 18:37:38 [11332] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
a803ca7c-a9b1-490f-9ab6-5c18bb81be3f: rc=26, reply=1, notify=0, exit=4203648
Oct 12 18:37:38 [11332] pm1 lrmd: info: log_execute: executing
- rsc:pgsql action:notify call_id:26
Oct 12 18:37:38 [11335] pm1 crmd: debug: run_graph: Transition
12 (Complete=1, Pending=2, Fired=2, Skipped=0, Incomplete=8,
Source=/var/lib/pacemaker/pengine/pe-input-106.bz2): In-progress
Oct 12 18:37:38 [31675] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:38 [31675] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31675-24-header
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31675-24-header
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31675-24-header
Oct 12 18:37:38 [31675] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31675-24-header
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31675-24-header
Oct 12 18:37:38 [31675] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31675-24-header
Oct 12 18:37:38 [31675] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
pgsql(pgsql)[31641]: 2015/10/12_18:37:38 DEBUG: notify: pre for stop
Oct 12 18:37:38 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31641 - exited with rc=0
Oct 12 18:37:38 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31641:stderr [ -- empty -- ]
Oct 12 18:37:38 [11332] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:31641:stdout [ -- empty -- ]
Oct 12 18:37:38 [11332] pm1 lrmd: info: log_finished: finished
- rsc:pgsql action:notify call_id:26 pid:31641 exit-code:0
exec-time:85ms queue-time:0ms
Oct 12 18:37:38 [11335] pm1 crmd: debug:
create_operation_update: send_direct_ack: Updating resource pgsql
after notify op complete (interval=0)
Oct 12 18:37:38 [11335] pm1 crmd: debug: send_direct_ack:
ACK'ing resource op pgsql_notify_0 from
47:12:0:27561a4e-40a7-4815-98c0-df6b485550dc: lrm_invoke-lrmd-1444664258-60
Oct 12 18:37:38 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664258-60 from pm1
Oct 12 18:37:38 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (47) confirmed on pm1 (rc=0)
Oct 12 18:37:38 [11335] pm1 crmd: notice: process_lrm_event:
Operation pgsql_notify_0: ok (node=pm1, call=26, rc=0, cib-update=0,
confirmed=true)
Oct 12 18:37:38 [11335] pm1 crmd: debug: run_graph: Transition
12 (Complete=2, Pending=1, Fired=0, Skipped=0, Incomplete=8,
Source=/var/lib/pacemaker/pengine/pe-input-106.bz2): In-progress
Oct 12 18:37:38 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664258-14 from pm2
Oct 12 18:37:38 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (45) confirmed on pm2 (rc=0)
Oct 12 18:37:38 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 18 fired and confirmed
Oct 12 18:37:38 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 15 fired and confirmed
Oct 12 18:37:38 [11335] pm1 crmd: debug: run_graph: Transition
12 (Complete=3, Pending=0, Fired=2, Skipped=0, Incomplete=6,
Source=/var/lib/pacemaker/pengine/pe-input-106.bz2): In-progress
Oct 12 18:37:38 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 2: stop pgsql_stop_0 on pm1 (local)
Oct 12 18:37:38 [11335] pm1 crmd: debug: do_lrm_rsc_op: Stopped
0 recurring operations in preparation for pgsql_stop_0
Oct 12 18:37:38 [11335] pm1 crmd: info: do_lrm_rsc_op:
Performing key=2:12:0:27561a4e-40a7-4815-98c0-df6b485550dc op=pgsql_stop_0
Oct 12 18:37:38 [11332] pm1 lrmd: debug: create_lrmd_cmd:
Setting flag to leave pid group on timeout and only kill action pid for
pgsql_stop_0
Oct 12 18:37:38 [11332] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
a803ca7c-a9b1-490f-9ab6-5c18bb81be3f: rc=27, reply=1, notify=0, exit=4203648
Oct 12 18:37:38 [11332] pm1 lrmd: info: log_execute: executing
- rsc:pgsql action:stop call_id:27
Oct 12 18:37:38 [11335] pm1 crmd: debug: run_graph: Transition
12 (Complete=5, Pending=1, Fired=1, Skipped=0, Incomplete=5,
Source=/var/lib/pacemaker/pengine/pe-input-106.bz2): In-progress
Oct 12 18:37:38 [31763] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:38 [31763] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31763-24-header
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31763-24-header
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31763-24-header
Oct 12 18:37:38 [31763] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31763-24-header
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31763-24-header
Oct 12 18:37:38 [31763] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31763-24-header
Oct 12 18:37:38 [31763] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:38 [31814] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:37:38 [31814] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-11314-31814-24-header
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-11314-31814-24-header
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-11314-31814-24-header
Oct 12 18:37:38 [31814] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-11314-31814-24-header
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-11314-31814-24-header
Oct 12 18:37:38 [31814] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-11314-31814-24-header
Oct 12 18:37:38 [31814] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:38 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31816
id=57d7759b-c643-4a76-8469-ba69bca69689
Oct 12 18:37:38 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31816-14)
Oct 12 18:37:38 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31816]
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:38 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:38 [31816] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:38 [31816] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:37:38 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bac5c0 for uid=0 gid=0 pid=31816
id=24991fbe-30fb-4805-86da-67eea46a0f0f
Oct 12 18:37:38 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-31816-11)
Oct 12 18:37:38 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [31816]
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting master-pgsql[pm1] = -INFINITY
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: master-pgsql=-INFINITY for pm1
Oct 12 18:37:38 [31816] pm1 crm_attribute: info: main: Update
master-pgsql=-INFINITY sent via attrd
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31816-14-header
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31816-14-header
Oct 12 18:37:38 [31816] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31816-14-header
Oct 12 18:37:38 [31816] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:38 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31816-14)
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31816-14) state:2
Oct 12 18:37:38 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31816-14-header
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31816-14-header
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31816-14-header
Oct 12 18:37:38 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-31816-11)
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-31816-11) state:2
Oct 12 18:37:38 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-31816-11-header
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-31816-11-header
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-31816-11-header
Oct 12 18:37:38 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31817
id=fd9ca7ea-d091-4786-9104-f0e6c6cb1d00
Oct 12 18:37:38 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31817-14)
Oct 12 18:37:38 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31817]
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:38 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:38 [31817] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:38 [31817] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:37:38 [11333] pm1 attrd: debug: crm_client_new:
Connecting 0x1bab000 for uid=0 gid=0 pid=31817
id=044c5a32-c07e-4c64-b3f9-a76a038322e4
Oct 12 18:37:38 [11333] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (11333-31817-11)
Oct 12 18:37:38 [11333] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [31817]
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:37:38 [11333] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-xlog-loc[pm1] = (null)
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-xlog-loc=(null) for pm1
Oct 12 18:37:38 [31817] pm1 crm_attribute: info: main: Update
pgsql-xlog-loc=<none> sent via attrd
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:38 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31817-14)
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31817-14) state:2
Oct 12 18:37:38 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31817-14-header
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31817-14-header
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31817-14-header
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31817-14-header
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31817-14-header
Oct 12 18:37:38 [31817] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31817-14-header
Oct 12 18:37:38 [31817] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:38 [11333] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11333-31817-11)
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11333-31817-11) state:2
Oct 12 18:37:38 [11333] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-11333-31817-11-header
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-11333-31817-11-header
Oct 12 18:37:38 [11333] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-11333-31817-11-header
pgsql(pgsql)[31729]: 2015/10/12_18:37:38 INFO: PostgreSQL is already
stopped.
Oct 12 18:37:38 [31828] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:37:38 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31828
id=54b39a15-6635-4272-b5ac-d79b54a10074
Oct 12 18:37:38 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31828-14)
Oct 12 18:37:38 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31828]
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31828] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31828] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31828] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31828] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:38 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:38 [31828] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:37:38 [31828] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:37:38 [31828] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:37:38 [31828] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:37:38 [31828] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:37:38 [31828] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:37:38 [31828] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:37:38 [31828] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:37:38 [31828] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:37:38 [31828] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:37:38 [31828] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:37:38 [31828] pm1 crm_mon: debug:
determine_op_status: pgsql_start_0 on pm1 returned 'unknown error'
(1) instead of the expected value: 'ok' (0)
Oct 12 18:37:38 [31828] pm1 crm_mon: warning:
unpack_rsc_op_failure: Processing failed op start for pgsql:1 on pm1:
unknown error (1)
Oct 12 18:37:38 [31828] pm1 crm_mon: debug:
determine_op_status: pgsql_start_0 on pm1 returned 'unknown error'
(1) instead of the expected value: 'ok' (0)
Oct 12 18:37:38 [31828] pm1 crm_mon: warning:
unpack_rsc_op_failure: Processing failed op start for pgsql:1 on pm1:
unknown error (1)
Oct 12 18:37:38 [31828] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:37:38 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31828-14)
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31828-14) state:2
Oct 12 18:37:38 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-11330-31828-14-header
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-11330-31828-14-header
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-11330-31828-14-header
Oct 12 18:37:38 [11330] pm1 cib: debug: crm_client_new:
Connecting 0x251cec0 for uid=0 gid=0 pid=31830
id=7822e1b5-1c01-4411-9e99-97300696efce
Oct 12 18:37:38 [11330] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (11330-31830-14)
Oct 12 18:37:38 [11330] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31830]
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:37:38 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:37:38 [31830] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:37:38 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:38 [11330] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/status//node_state[@id='1']//transient_attributes//nvpair[@name='pgsql-status']
(/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3])
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
find_nvpair_attr_delegate: Match <nvpair id="status-1-pgsql-status"
name="pgsql-status" value="STOP"/>
Oct 12 18:37:38 [31830] pm1 crm_attribute: info: main: Read
pgsql-status=STOP
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-11330-31830-14-header
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-11330-31830-14-header
Oct 12 18:37:38 [31830] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-11330-31830-14-header
Oct 12 18:37:38 [31830] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:37:38 [11332] pm1 lrmd: debug: operation_finished:
pgsql_stop_0:31729 - exited with rc=0
Oct 12 18:37:38 [11332] pm1 lrmd: debug: operation_finished:
pgsql_stop_0:31729:stderr [ -- empty -- ]
Oct 12 18:37:38 [11332] pm1 lrmd: debug: operation_finished:
pgsql_stop_0:31729:stdout [ -- empty -- ]
Oct 12 18:37:38 [11332] pm1 lrmd: info: log_finished: finished
- rsc:pgsql action:stop call_id:27 pid:31729 exit-code:0 exec-time:145ms
queue-time:0ms
Oct 12 18:37:38 [11335] pm1 crmd: debug:
create_operation_update: do_update_resource: Updating resource pgsql
after stop op complete (interval=0)
Oct 12 18:37:38 [11335] pm1 crmd: notice: process_lrm_event:
Operation pgsql_stop_0: ok (node=pm1, call=27, rc=0, cib-update=55,
confirmed=true)
Oct 12 18:37:38 [11335] pm1 crmd: debug:
update_history_cache: Updating history for 'pgsql' with stop op
Oct 12 18:37:38 [11330] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/55)
Oct 12 18:37:38 [11330] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (11330-31830-14)
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(11330-31830-14) state:2
Oct 12 18:37:38 [11330] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-11330-31830-14-header
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-11330-31830-14-header
Oct 12 18:37:38 [11330] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-11330-31830-14-header
Oct 12 18:37:38 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:38 [11330] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.43 2
Oct 12 18:37:38 [11330] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.44 (null)
Oct 12 18:37:38 [11330] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=44
Oct 12 18:37:38 [11330] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='pgsql']/lrm_rsc_op[@id='pgsql_last_0']:
@operation_key=pgsql_stop_0, @operation=stop,
@transition-key=2:12:0:27561a4e-40a7-4815-98c0-df6b485550dc,
@transition-magic=0:0;2:12:0:27561a4e-40a7-4815-98c0-df6b485550dc,
@call-id=27, @rc-code=0, @last-run=1444664258,
@last-rc-change=1444664258, @exec-time=145
Oct 12 18:37:38 [11330] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/55, version=0.19.44)
Oct 12 18:37:38 [11331] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.44 to 0.19.43
Oct 12 18:37:38 [11335] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.43 -> 0.19.44 (S_TRANSITION_ENGINE)
Oct 12 18:37:38 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_stop_0 (2) confirmed on pm1 (rc=0)
Oct 12 18:37:38 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 16 fired and confirmed
Oct 12 18:37:38 [11335] pm1 crmd: debug: run_graph: Transition
12 (Complete=6, Pending=0, Fired=1, Skipped=0, Incomplete=4,
Source=/var/lib/pacemaker/pengine/pe-input-106.bz2): In-progress
Oct 12 18:37:38 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 19 fired and confirmed
Oct 12 18:37:38 [11335] pm1 crmd: debug: run_graph: Transition
12 (Complete=7, Pending=0, Fired=1, Skipped=0, Incomplete=3,
Source=/var/lib/pacemaker/pengine/pe-input-106.bz2): In-progress
Oct 12 18:37:38 [11335] pm1 crmd: notice: te_rsc_command:
Initiating action 46: notify pgsql_post_notify_stop_0 on pm2
Oct 12 18:37:38 [11335] pm1 crmd: debug: run_graph: Transition
12 (Complete=8, Pending=1, Fired=1, Skipped=0, Incomplete=2,
Source=/var/lib/pacemaker/pengine/pe-input-106.bz2): In-progress
Oct 12 18:37:38 [11335] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664258-15 from pm2
Oct 12 18:37:38 [11335] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (46) confirmed on pm2 (rc=0)
Oct 12 18:37:38 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 20 fired and confirmed
Oct 12 18:37:38 [11335] pm1 crmd: debug: te_pseudo_action:
Pseudo action 3 fired and confirmed
Oct 12 18:37:38 [11335] pm1 crmd: debug: run_graph: Transition
12 (Complete=9, Pending=0, Fired=2, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-106.bz2): In-progress
Oct 12 18:37:38 [11335] pm1 crmd: notice: run_graph: Transition
12 (Complete=11, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-106.bz2): Complete
Oct 12 18:37:38 [11335] pm1 crmd: debug: te_graph_trigger:
Transition 12 is now complete
Oct 12 18:37:38 [11335] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:37:38 [11335] pm1 crmd: debug: notify_crmd:
Transition 12 status: done - <null>
Oct 12 18:37:38 [11335] pm1 crmd: debug: s_crmd_fsa: Processing
I_TE_SUCCESS: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:37:38 [11335] pm1 crmd: info: do_log: FSA: Input
I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE
Oct 12 18:37:38 [11335] pm1 crmd: notice:
do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [
input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:37:38 [11335] pm1 crmd: debug:
do_state_transition: Starting PEngine Recheck Timer
Oct 12 18:37:38 [11335] pm1 crmd: debug: crm_timer_start:
Started PEngine Recheck Timer (I_PE_CALC:900000ms), src=100
Oct 12 18:37:43 [11330] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:37:43 [11330] pm1 cib: info: cib_process_ping:
Reporting our current digest to pm1: 4c044375f453b997641f54a618be6abe
for 0.19.44 (0x2363f00 0)
Oct 12 18:37:49 [11335] pm1 crmd: debug: throttle_cib_load:
cib load: 0.005000 (15 ticks in 30s)
Oct 12 18:37:49 [11335] pm1 crmd: debug: throttle_load_avg:
Current load is 0.070000 (full: 0.07 0.07 0.05 2/115 31831)
Oct 12 18:37:49 [11335] pm1 crmd: debug: throttle_io_load:
Current IO load is 0.000000
On 12.10.2015 18:06, Nikolay Popov wrote:
> Hi, Andrey.
>
> Check parameter /postgrresql.conf/ (unix_socket_directories) and
> RA pqsql (OCF_RESKEY_socketdir=/tmp)
>
> > stderr: psql: could not connect to server: No such file or directory
> > stderr: Is the server running locally and accepting
> > stderr: connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
> > stderr: WARNING: Can't get PostgreSQL recovery status. rc=2
>
> 12.10.2015 17:44, Andrey Rubik пишет:
>> Thanks for reply.
>>
>> Now I testing only Master/Slave configuration, but before i tried
>> full configuration with same results:
>>
>> pcs cluster cib pgsql_cfg
>>
>> pcs -f pgsql_cfg property set no-quorum-policy="ignore"
>> pcs -f pgsql_cfg property set stonith-enabled="false"
>> pcs -f pgsql_cfg resource defaults resource-stickiness="INFINITY"
>> pcs -f pgsql_cfg resource defaults migration-threshold="2"
>>
>> pcs -f pgsql_cfg resource create vip-master IPaddr2 \
>> ip="10.200.200.225" \
>> nic="eth0" \
>> cidr_netmask="24" \
>> op start timeout="60s" interval="0s" on-fail="restart" \
>> op monitor timeout="60s" interval="10s" on-fail="restart" \
>> op stop timeout="60s" interval="0s" on-fail="block"
>>
>> pcs -f pgsql_cfg resource create vip-rep IPaddr2 \
>> ip="10.200.200.224" \
>> nic="eth0" \
>> cidr_netmask="24" \
>> meta migration-threshold="0" \
>> op start timeout="60s" interval="0s" on-fail="stop" \
>> op monitor timeout="60s" interval="10s" on-fail="restart" \
>> op stop timeout="60s" interval="0s" on-fail="ignore"
>>
>> pcs -f pgsql_cfg resource create pgsql pgsql \
>> pgctl="/usr/pgsql-9.3/bin/pg_ctl" \
>> psql="/usr/pgsql-9.3/bin/psql" \
>> pgdata="/var/lib/pgsql/9.3/data/" \
>> tmpdir="/var/lib/pgsql/9.3/tmp/" \
>> rep_mode="sync" \
>> node_list="pg1 pg2" \
>> restore_command="cp /var/lib/pgsql/9.3/backups/%f %p" \
>> primary_conninfo_opt="keepalives_idle=60 keepalives_interval=5
>> keepalives_count=5" \
>> master_ip="10.200.200.225" \
>> restart_on_promote='false' \
>> op start timeout="60s" interval="0s" on-fail="restart" \
>> op monitor timeout="60s" interval="4s" on-fail="restart" \
>> op monitor timeout="60s" interval="3s" on-fail="restart"
>> role="Master" \
>> op promote timeout="60s" interval="0s" on-fail="restart" \
>> op demote timeout="60s" interval="0s" on-fail="stop" \
>> op stop timeout="60s" interval="0s" on-fail="block" \
>> op notify timeout="60s" interval="0s"
>>
>> pcs -f pgsql_cfg resource master msPostgresql pgsql \
>> master-max=1 master-node-max=1 clone-max=2 clone-node-max=1
>> notify=true
>>
>> pcs -f pgsql_cfg resource group add master-group vip-master vip-rep
>>
>> pcs -f pgsql_cfg constraint colocation add master-group with Master
>> msPostgresql INFINITY
>> pcs -f pgsql_cfg constraint order promote msPostgresql then start
>> master-group symmetrical=false score=INFINITY
>> pcs -f pgsql_cfg constraint order demote msPostgresql then stop
>> master-group symmetrical=false score=0
>>
>> pcs cluster cib-push pgsql_cfg
>>
>>
>>
>> crm_mon in this case:
>>
>> Stack: corosync
>> Current DC: pm2 (version 1.1.13-a14efad) - partition with quorum
>> 2 nodes and 4 resources configured
>>
>> Online: [ pm1 pm2 ]
>>
>> Full list of resources:
>>
>> Master/Slave Set: msPostgresql [pgsql]
>> Slaves: [ pm1 pm2 ]
>> Resource Group: master-group
>> vip-master (ocf::heartbeat:IPaddr2): Stopped
>> vip-rep (ocf::heartbeat:IPaddr2): Stopped
>>
>> Node Attributes:
>> * Node pm1:
>> + master-pgsql : -INFINITY
>> + pgsql-status : HS:alone
>> + pgsql-xlog-loc : 0000000000000000
>> * Node pm2:
>> + pgsql-status : HS:alone
>> + pgsql-xlog-loc : 0000000000000000
>>
>> Migration Summary:
>> * Node pm2:
>> * Node pm1:
>>
>>
>>
>>
>>
>>
>> On 12.10.2015 17:36, Nikolay Popov wrote:
>>> Hi, Andrey.
>>>
>>> You added VIP?
>>> That is a VIP (master_ip="10.200.200.233")?
>>>
>>>
>>>
>>> 12.10.2015 17:23, Andrey Rubik пишет:
>>>> Hi all.
>>>>
>>>> I'm trying to configure pgsql Master/Slave set with 2 nodes.
>>>>
>>>> I'm get successful installation Master/Slave without pacemaker.
>>>>
>>>> Pacemaker version:
>>>>
>>>> pacemakerd --features
>>>> Pacemaker 1.1.13 (Build: a14efad)
>>>> Supporting v3.0.10: generated-manpages agent-manpages ascii-docs
>>>> publican-docs ncurses libqb-logging libqb-ipc upstart systemd
>>>> nagios corosync-native atomic-attrd acls
>>>>
>>>> That's my configuration:
>>>>
>>>> pcs cluster cib pgsql_cfg
>>>>
>>>> pcs -f pgsql_cfg property set no-quorum-policy="ignore"
>>>> pcs -f pgsql_cfg property set stonith-enabled="false"
>>>>
>>>> pcs -f pgsql_cfg resource create pgsql pgsql \
>>>> pgctl="/usr/pgsql-9.3/bin/pg_ctl" \
>>>> psql="/usr/pgsql-9.3/bin/psql" \
>>>> pgdata="/var/lib/pgsql/9.3/data/" \
>>>> tmpdir="/var/lib/pgsql/9.3/tmp/" \
>>>> rep_mode="sync" \
>>>> node_list="pg1 pg2" \
>>>> restore_command="cp /var/lib/pgsql/9.3/backups/%f %p" \
>>>> primary_conninfo_opt="keepalives_idle=60 keepalives_interval=5
>>>> keepalives_count=5" \
>>>> master_ip="10.200.200.233" \
>>>> restart_on_promote='true' \
>>>> op start timeout="60s" interval="0s" on-fail="restart" \
>>>> op monitor timeout="60s" interval="4s" on-fail="restart" \
>>>> op monitor timeout="60s" interval="3s" on-fail="restart"
>>>> role="Master" \
>>>> op promote timeout="60s" interval="0s" on-fail="restart" \
>>>> op demote timeout="60s" interval="0s" on-fail="stop" \
>>>> op stop timeout="60s" interval="0s" on-fail="block" \
>>>> op notify timeout="60s" interval="0s"
>>>>
>>>> pcs -f pgsql_cfg resource master msPostgresql pgsql \
>>>> master-max=1 master-node-max=1 clone-max=2 clone-node-max=1
>>>> notify=true
>>>>
>>>> pcs cluster cib-push pgsql_cfg
>>>>
>>>>
>>>>
>>>> crm_mon:
>>>>
>>>> Stack: corosync
>>>> Current DC: pm2 (version 1.1.13-a14efad) - partition with quorum
>>>> 2 nodes and 2 resources configured
>>>>
>>>> Online: [ pm1 pm2 ]
>>>>
>>>> Full list of resources:
>>>>
>>>> Master/Slave Set: msPostgresql [pgsql]
>>>> Slaves: [ pm1 pm2 ]
>>>>
>>>> Node Attributes:
>>>> * Node pm1:
>>>> + master-pgsql : -INFINITY
>>>> + pgsql-data-status : LATEST
>>>> + pgsql-status : HS:alone
>>>> + pgsql-xlog-loc : 0000000000000000
>>>> * Node pm2:
>>>> + master-pgsql : -INFINITY
>>>> + pgsql-data-status : LATEST
>>>> + pgsql-master-baseline : 0000000000000000
>>>> + pgsql-status : HS:alone
>>>> + pgsql-xlog-loc : 0000000000000000
>>>>
>>>> Migration Summary:
>>>> * Node pm2:
>>>> * Node pm1:
>>>>
>>>>
>>>>
>>>> /var/log/pacemaker.log:
>>>>
>>>> WARNING: Can't get pg1 xlog location.
>>>> WARNING: Can't get pg2 xlog location.
>>>> WARNING: Can't get pg1 xlog location.
>>>> WARNING: Can't get pg2 xlog location.
>>>> .....
>>>>
>>>> Its very strange messages since i have pgsql-xlog-loc in crm_mon.
>>>>
>>>>
>>>>
>>>>
>>>> If I try set master manually it fails and then stop pm1 node:
>>>>
>>>> crm_resource -N pm1 -r pgsql --force-promote
>>>>
>>>> Operation promote for pgsql:0 (ocf:heartbeat:pgsql) returned 0
>>>> > stderr: INFO: Creating /var/lib/pgsql/9.3/tmp//PGSQL.lock.
>>>> > stderr: INFO: My master baseline : 0000000000000000.
>>>> > stderr: INFO: Restarting PostgreSQL instead of promote.
>>>> > stderr: INFO: waiting for server to shut down.... done server
>>>> stopped
>>>> > stderr: INFO: PostgreSQL is down
>>>> > stderr: INFO: server starting
>>>> > stderr: INFO: PostgreSQL start command sent.
>>>> > stderr: psql: could not connect to server: No such file or
>>>> directory
>>>> > stderr: Is the server running locally and accepting
>>>> > stderr: connections on Unix domain socket
>>>> "/tmp/.s.PGSQL.5432"?
>>>> > stderr: WARNING: Can't get PostgreSQL recovery status. rc=2
>>>> > stderr: WARNING: Connection error (connection to the server
>>>> went bad and the session was not interactive) occurred while
>>>> executing the psql command.
>>>> > stderr: DEBUG: PostgreSQL still hasn't started yet. Waiting...
>>>> > stderr: DEBUG: PostgreSQL is running as a primary.
>>>> > stderr: INFO: PostgreSQL is started.
>>>> > stderr: INFO: Changing pgsql-status on pm1 : HS:alone->PRI.
>>>>
>>>> crm_mon:
>>>>
>>>> Stack: corosync
>>>> Current DC: pm2 (version 1.1.13-a14efad) - partition with quorum
>>>> 2 nodes and 2 resources configured
>>>>
>>>> Online: [ pm1 pm2 ]
>>>>
>>>> Full list of resources:
>>>>
>>>> Master/Slave Set: msPostgresql [pgsql]
>>>> Slaves: [ pm2 ]
>>>> Stopped: [ pm1 ]
>>>>
>>>> Node Attributes:
>>>> * Node pm1:
>>>> + master-pgsql : -INFINITY
>>>> + master-pgsql:0 : 1000
>>>> + pgsql-data-status : LATEST
>>>> + pgsql-status : STOP
>>>> * Node pm2:
>>>> + master-pgsql : -INFINITY
>>>> + pgsql-data-status : LATEST
>>>> + pgsql-master-baseline : 0000000000000000
>>>> + pgsql-status : HS:alone
>>>> + pgsql-xlog-loc : 0000000000000000
>>>>
>>>> Migration Summary:
>>>> * Node pm2:
>>>> * Node pm1:
>>>> pgsql: migration-threshold=1000000 fail-count=1000000
>>>> last-failure='Mon Oct 12 17:12:04 2015'
>>>>
>>>> Failed Actions:
>>>> * pgsql_start_0 on pm1 'unknown error' (1): call=47,
>>>> status=complete, exitreason='My data may be inconsistent. You have
>>>> to remove /var/lib/pgsql/9.3/tmp//PGSQL.lock file to force start.',
>>>> last-rc-change='Mon Oct 12 17:12:04 2015', queued=0ms, exec=170ms
>>>>
>>>>
>>>>
>>>> That is verbose variant:
>>>> rm -f /var/lib/pgsql/9.3/tmp//PGSQL.lock
>>>> crm_resource -N pm1 -r pgsql --force-promote -V
>>>> Operation promote for pgsql:0 (ocf:heartbeat:pgsql) returned 0
>>>> > stderr: + 17:14:11: 22: '[' -x /sbin/runuser ']'
>>>> > stderr: + 17:14:11: 23: SU=runuser
>>>> > stderr: + 17:14:11: 45: OCF_RESKEY_pgctl_default=/usr/bin/pg_ctl
>>>> > stderr: + 17:14:11: 46: OCF_RESKEY_psql_default=/usr/bin/psql
>>>> > stderr: + 17:14:11: 47:
>>>> OCF_RESKEY_pgdata_default=/var/lib/pgsql/data
>>>> > stderr: + 17:14:11: 48: OCF_RESKEY_pgdba_default=postgres
>>>> > stderr: + 17:14:11: 49: OCF_RESKEY_pghost_default=
>>>> > stderr: + 17:14:11: 50: OCF_RESKEY_pgport_default=5432
>>>> > stderr: + 17:14:11: 51: OCF_RESKEY_pglibs_default=/usr/lib
>>>> > stderr: + 17:14:11: 52: OCF_RESKEY_start_opt_default=
>>>> > stderr: + 17:14:11: 53: OCF_RESKEY_pgdb_default=template1
>>>> > stderr: + 17:14:11: 54: OCF_RESKEY_logfile_default=/dev/null
>>>> > stderr: + 17:14:11: 55: OCF_RESKEY_stop_escalate_default=30
>>>> > stderr: + 17:14:11: 56: OCF_RESKEY_monitor_user_default=
>>>> > stderr: + 17:14:11: 57: OCF_RESKEY_monitor_password_default=
>>>> > stderr: + 17:14:11: 58: OCF_RESKEY_monitor_sql_default='select
>>>> now();'
>>>> > stderr: + 17:14:11: 59:
>>>> OCF_RESKEY_check_wal_receiver_default=false
>>>> > stderr: + 17:14:11: 61: OCF_RESKEY_rep_mode_default=none
>>>> > stderr: + 17:14:11: 62: OCF_RESKEY_node_list_default=
>>>> > stderr: + 17:14:11: 63: OCF_RESKEY_restore_command_default=
>>>> > stderr: + 17:14:11: 64:
>>>> OCF_RESKEY_archive_cleanup_command_default=
>>>> > stderr: + 17:14:11: 65: OCF_RESKEY_recovery_end_command_default=
>>>> > stderr: + 17:14:11: 66: OCF_RESKEY_master_ip_default=
>>>> > stderr: + 17:14:11: 67: OCF_RESKEY_repuser_default=postgres
>>>> > stderr: + 17:14:11: 68: OCF_RESKEY_primary_conninfo_opt_default=
>>>> > stderr: + 17:14:11: 69:
>>>> OCF_RESKEY_restart_on_promote_default=false
>>>> > stderr: + 17:14:11: 70:
>>>> OCF_RESKEY_tmpdir_default=/var/lib/pgsql/tmp
>>>> > stderr: + 17:14:11: 71: OCF_RESKEY_xlog_check_count_default=3
>>>> > stderr: + 17:14:11: 72: OCF_RESKEY_crm_attr_timeout_default=5
>>>> > stderr: + 17:14:11: 73:
>>>> OCF_RESKEY_stop_escalate_in_slave_default=30
>>>> > stderr: + 17:14:11: 74: OCF_RESKEY_replication_slot_name_default=
>>>> > stderr: + 17:14:11: 76: : /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: + 17:14:11: 77: : /usr/pgsql-9.3/bin/psql
>>>> > stderr: + 17:14:11: 78: : /var/lib/pgsql/9.3/data/
>>>> > stderr: + 17:14:11: 79: : postgres
>>>> > stderr: + 17:14:11: 80: :
>>>> > stderr: + 17:14:11: 81: : 5432
>>>> > stderr: + 17:14:11: 82: : /usr/lib
>>>> > stderr: + 17:14:11: 83: : /var/lib/pgsql/9.3/data//postgresql.conf
>>>> > stderr: + 17:14:11: 84: :
>>>> > stderr: + 17:14:11: 85: : template1
>>>> > stderr: + 17:14:11: 86: : /dev/null
>>>> > stderr: + 17:14:11: 87: : 30
>>>> > stderr: + 17:14:11: 88: :
>>>> > stderr: + 17:14:11: 89: :
>>>> > stderr: + 17:14:11: 90: : select 'now();'
>>>> > stderr: + 17:14:11: 91: : false
>>>> > stderr: + 17:14:11: 94: : sync
>>>> > stderr: + 17:14:11: 95: : pg1 pg2
>>>> > stderr: + 17:14:11: 96: : cp /var/lib/pgsql/9.3/backups/%f %p
>>>> > stderr: + 17:14:11: 97: :
>>>> > stderr: + 17:14:11: 98: :
>>>> > stderr: + 17:14:11: 99: : 10.200.200.233
>>>> > stderr: + 17:14:11: 100: : postgres
>>>> > stderr: + 17:14:11: 101: : keepalives_idle=60
>>>> keepalives_interval=5 keepalives_count=5
>>>> > stderr: + 17:14:11: 102: : true
>>>> > stderr: + 17:14:11: 103: : /var/lib/pgsql/9.3/tmp/
>>>> > stderr: + 17:14:11: 104: : 3
>>>> > stderr: + 17:14:11: 105: : 5
>>>> > stderr: + 17:14:11: 106: : 30
>>>> > stderr: + 17:14:11: 107: :
>>>> > stderr: + 17:14:11: 1935: '[' 1 -ne 1 ']'
>>>> > stderr: + 17:14:11: 1942:
>>>> PIDFILE=/var/lib/pgsql/9.3/data//postmaster.pid
>>>> > stderr: + 17:14:11: 1943:
>>>> BACKUPLABEL=/var/lib/pgsql/9.3/data//backup_label
>>>> > stderr: ++ 17:14:11: 1944: echo pgsql:0
>>>> > stderr: ++ 17:14:11: 1944: cut -d : -f 1
>>>> > stderr: + 17:14:11: 1944: RESOURCE_NAME=pgsql
>>>> > stderr: + 17:14:11: 1945:
>>>> PGSQL_WAL_RECEIVER_STATUS_ATTR=pgsql-receiver-status
>>>> > stderr: + 17:14:11: 1946:
>>>> RECOVERY_CONF=/var/lib/pgsql/9.3/data//recovery.conf
>>>> > stderr: ++ 17:14:11: 1947: ocf_local_nodename
>>>> > stderr: ++ 17:14:11: 1947: tr '[A-Z]' '[a-z]'
>>>> > stderr: ++ 17:14:11: ocf_local_nodename:570: which pacemakerd
>>>> > stderr: ++ 17:14:11: ocf_local_nodename:571: '[' 0 -eq 0 ']'
>>>> > stderr: +++ 17:14:11: ocf_local_nodename:572: pacemakerd '-$'
>>>> > stderr: +++ 17:14:12: ocf_local_nodename:572: grep 'Pacemaker .*'
>>>> > stderr: +++ 17:14:12: ocf_local_nodename:572: awk '{ print $2 }'
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:572: local version=1.1.13
>>>> > stderr: +++ 17:14:12: ocf_local_nodename:573: echo 1.1.13
>>>> > stderr: +++ 17:14:12: ocf_local_nodename:573: awk -F- '{ print
>>>> $1 }'
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:573: version=1.1.13
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:574: ocf_version_cmp
>>>> 1.1.13 1.1.8
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:543: ocf_is_ver 1.1.13
>>>> > stderr: ++ 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>>>> > stderr: ++ 17:14:12: ocf_is_ver:513: echo 1.1.13
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:544: ocf_is_ver 1.1.8
>>>> > stderr: ++ 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>>>> > stderr: ++ 17:14:12: ocf_is_ver:513: echo 1.1.8
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:545: local v1=1.1.13
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:546: local v2=1.1.8
>>>> > stderr: +++ 17:14:12: ocf_version_cmp:547: ocf_ver_level 1.1.13
>>>> > stderr: +++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>>>> > stderr: +++ 17:14:12: ocf_ver_level:521: echo 1.1.13
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:547: local v1_level=3
>>>> > stderr: +++ 17:14:12: ocf_version_cmp:548: ocf_ver_level 1.1.8
>>>> > stderr: +++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>>>> > stderr: +++ 17:14:12: ocf_ver_level:521: echo 1.1.8
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:548: local v2_level=3
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:549: local level_diff
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:550: '[' 3 -lt 3 ']'
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:553: '[' 3 -gt 3 ']'
>>>> > stderr: +++ 17:14:12: ocf_version_cmp:557: ocf_ver2num 1.1.13
>>>> > stderr: +++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>>>> > stderr: {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>>>> > stderr: '
>>>> > stderr: +++ 17:14:12: ocf_ver2num:516: echo 1.1.13
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:557: v1=1001013
>>>> > stderr: +++ 17:14:12: ocf_version_cmp:558: ocf_ver2num 1.1.8
>>>> > stderr: +++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>>>> > stderr: {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>>>> > stderr: '
>>>> > stderr: +++ 17:14:12: ocf_ver2num:516: echo 1.1.8
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:558: v2=1001008
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:559: '[' 1001013 -eq
>>>> 1001008 ']'
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:561: '[' 1001013 -lt
>>>> 1001008 ']'
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:564: return 2
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:575: '[' 2 -eq 2 ']'
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:576: which crm_node
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:577: '[' 0 -eq 0 ']'
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:578: crm_node -n
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:579: return
>>>> > stderr: + 17:14:12: 1947: NODENAME=pm1
>>>> > stderr: + 17:14:12: 1949: is_replication
>>>> > stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a
>>>> sync '!=' slave ']'
>>>> > stderr: + 17:14:12: is_replication:1288: return 0
>>>> > stderr: + 17:14:12: 1950:
>>>> REP_MODE_CONF=/var/lib/pgsql/9.3/tmp//rep_mode.conf
>>>> > stderr: + 17:14:12: 1951:
>>>> PGSQL_LOCK=/var/lib/pgsql/9.3/tmp//PGSQL.lock
>>>> > stderr: + 17:14:12: 1952:
>>>> XLOG_NOTE_FILE=/var/lib/pgsql/9.3/tmp//xlog_note
>>>> > stderr: + 17:14:12: 1954: CRM_MASTER='/usr/sbin/crm_master -l
>>>> reboot'
>>>> > stderr: + 17:14:12: 1955:
>>>> CRM_ATTR_REBOOT='/usr/sbin/crm_attribute -l reboot'
>>>> > stderr: + 17:14:12: 1956:
>>>> CRM_ATTR_FOREVER='/usr/sbin/crm_attribute -l forever'
>>>> > stderr: + 17:14:12: 1957: CRM_FAILCOUNT=/usr/sbin/crm_failcount
>>>> > stderr: + 17:14:12: 1959: CAN_NOT_PROMOTE=-INFINITY
>>>> > stderr: + 17:14:12: 1960: CAN_PROMOTE=100
>>>> > stderr: + 17:14:12: 1961: PROMOTE_ME=1000
>>>> > stderr: + 17:14:12: 1963: CHECK_MS_SQL='select
>>>> pg_is_in_recovery()'
>>>> > stderr: + 17:14:12: 1964: CHECK_XLOG_LOC_SQL='select
>>>> pg_last_xlog_replay_location(),pg_last_xlog_receive_location()'
>>>> > stderr: + 17:14:12: 1965: CHECK_REPLICATION_STATE_SQL='select
>>>> application_name,upper(state),upper(sync_state) from
>>>> pg_stat_replication'
>>>> > stderr: + 17:14:12: 1967: PGSQL_STATUS_ATTR=pgsql-status
>>>> > stderr: + 17:14:12: 1968: PGSQL_DATA_STATUS_ATTR=pgsql-data-status
>>>> > stderr: + 17:14:12: 1969: PGSQL_XLOG_LOC_NAME=pgsql-xlog-loc
>>>> > stderr: + 17:14:12: 1970:
>>>> PGSQL_MASTER_BASELINE=pgsql-master-baseline
>>>> > stderr: ++ 17:14:12: 1972: echo pg1 pg2
>>>> > stderr: ++ 17:14:12: 1972: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:12: 1972: NODE_LIST='pg1 pg2'
>>>> > stderr: + 17:14:12: 1973: RE_CONTROL_SLAVE=false
>>>> > stderr: + 17:14:12: 1976: case "$1" in
>>>> > stderr: + 17:14:12: 1984: pgsql_validate_all
>>>> > stderr: + 17:14:12: pgsql_validate_all:1749: local version
>>>> > stderr: + 17:14:12: pgsql_validate_all:1750: local check_config_rc
>>>> > stderr: + 17:14:12: pgsql_validate_all:1751: local rep_mode_string
>>>> > stderr: + 17:14:12: pgsql_validate_all:1752: local
>>>> socket_directories
>>>> > stderr: ++ 17:14:12: pgsql_validate_all:1754: cat
>>>> /var/lib/pgsql/9.3/data//PG_VERSION
>>>> > stderr: + 17:14:12: pgsql_validate_all:1754: version=9.3
>>>> > stderr: + 17:14:12: pgsql_validate_all:1756: check_binary2
>>>> /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: + 17:14:12: check_binary2:1724: have_binary
>>>> /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: + 17:14:12: have_binary:68: '[' '' = 1 ']'
>>>> > stderr: ++ 17:14:12: have_binary:71: echo
>>>> /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: ++ 17:14:12: have_binary:71: sed -e 's/ -.*//'
>>>> > stderr: + 17:14:12: have_binary:71: local
>>>> bin=/usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: ++ 17:14:12: have_binary:72: which
>>>> /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: + 17:14:12: have_binary:72: test -x
>>>> /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: + 17:14:12: check_binary2:1728: return 0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1757: check_binary2
>>>> /usr/pgsql-9.3/bin/psql
>>>> > stderr: + 17:14:12: check_binary2:1724: have_binary
>>>> /usr/pgsql-9.3/bin/psql
>>>> > stderr: + 17:14:12: have_binary:68: '[' '' = 1 ']'
>>>> > stderr: ++ 17:14:12: have_binary:71: echo /usr/pgsql-9.3/bin/psql
>>>> > stderr: ++ 17:14:12: have_binary:71: sed -e 's/ -.*//'
>>>> > stderr: + 17:14:12: have_binary:71: local
>>>> bin=/usr/pgsql-9.3/bin/psql
>>>> > stderr: ++ 17:14:12: have_binary:72: which /usr/pgsql-9.3/bin/psql
>>>> > stderr: + 17:14:12: have_binary:72: test -x
>>>> /usr/pgsql-9.3/bin/psql
>>>> > stderr: + 17:14:12: check_binary2:1728: return 0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1761: check_config
>>>> /var/lib/pgsql/9.3/data//postgresql.conf
>>>> > stderr: + 17:14:12: check_config:1732: local rc=0
>>>> > stderr: + 17:14:12: check_config:1734: '[' '!' -f
>>>> /var/lib/pgsql/9.3/data//postgresql.conf ']'
>>>> > stderr: + 17:14:12: check_config:1744: return 0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1762: check_config_rc=0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1763: '[' 0 -eq 2 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1764: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1765: ocf_version_cmp
>>>> 9.3 9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:543: ocf_is_ver 9.3
>>>> > stderr: + 17:14:12: ocf_is_ver:513: echo 9.3
>>>> > stderr: + 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>>>> > stderr: + 17:14:12: ocf_version_cmp:544: ocf_is_ver 9.3
>>>> > stderr: + 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>>>> > stderr: + 17:14:12: ocf_is_ver:513: echo 9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:545: local v1=9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:546: local v2=9.3
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:547: ocf_ver_level 9.3
>>>> > stderr: ++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>>>> > stderr: ++ 17:14:12: ocf_ver_level:521: echo 9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:547: local v1_level=2
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:548: ocf_ver_level 9.3
>>>> > stderr: ++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>>>> > stderr: ++ 17:14:12: ocf_ver_level:521: echo 9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:548: local v2_level=2
>>>> > stderr: + 17:14:12: ocf_version_cmp:549: local level_diff
>>>> > stderr: + 17:14:12: ocf_version_cmp:550: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:12: ocf_version_cmp:553: '[' 2 -gt 2 ']'
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:557: ocf_ver2num 9.3
>>>> > stderr: ++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>>>> > stderr: {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>>>> > stderr: '
>>>> > stderr: ++ 17:14:12: ocf_ver2num:516: echo 9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:557: v1=9003
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:558: ocf_ver2num 9.3
>>>> > stderr: ++ 17:14:12: ocf_ver2num:516: echo 9.3
>>>> > stderr: ++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>>>> > stderr: {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>>>> > stderr: '
>>>> > stderr: + 17:14:12: ocf_version_cmp:558: v2=9003
>>>> > stderr: + 17:14:12: ocf_version_cmp:559: '[' 9003 -eq 9003 ']'
>>>> > stderr: + 17:14:12: ocf_version_cmp:560: return 1
>>>> > stderr: + 17:14:12: pgsql_validate_all:1766: '[' 1 -eq 0 ']'
>>>> > stderr: ++ 17:14:12: pgsql_validate_all:1770: get_pgsql_param
>>>> unix_socket_directories
>>>> > stderr: ++ 17:14:12: get_pgsql_param:32: local param_name
>>>> > stderr: ++ 17:14:12: get_pgsql_param:34:
>>>> param_name=unix_socket_directories
>>>> > stderr: ++ 17:14:12: get_pgsql_param:39: perl_code='if
>>>> (/^\s*unix_socket_directories[\s=]+\s*(.*)$/) {
>>>> > stderr: $dir=$1;
>>>> > stderr: $dir =~ s/\s*\#.*//;
>>>> > stderr: $dir =~ s/^'\''(\S*)'\''/$1/;
>>>> > stderr: print $dir;}'
>>>> > stderr: ++ 17:14:12: get_pgsql_param:41: perl -ne 'if
>>>> (/^\s*unix_socket_directories[\s=]+\s*(.*)$/) {
>>>> > stderr: $dir=$1;
>>>> > stderr: $dir =~ s/\s*\#.*//;
>>>> > stderr: $dir =~ s/^'\''(\S*)'\''/$1/;
>>>> > stderr: print $dir;}'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1770: socket_directories=
>>>> > stderr: + 17:14:12: pgsql_validate_all:1771: '[' -n '' ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1782: getent passwd
>>>> postgres
>>>> > stderr: + 17:14:12: pgsql_validate_all:1783: '[' '!' 0 -eq 0 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1788: ocf_is_probe
>>>> > stderr: + 17:14:12: ocf_is_probe:490: '[' promote = monitor -a
>>>> 0 = 0 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1791: runasowner 'test
>>>> -w /var/lib/pgsql/9.3/data/'
>>>> > stderr: + 17:14:12: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:12: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:12: runasowner:469: local var
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; test -w
>>>> /var/lib/pgsql/9.3/data/'
>>>> > stderr: + 17:14:12: ocf_run:413: local rc
>>>> > stderr: + 17:14:12: ocf_run:414: local output
>>>> > stderr: + 17:14:12: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:12: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:417: local var
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:426: echo -err
>>>> > stderr: ++ 17:14:12: ocf_run:426: sed -e s/-//g
>>>> > stderr: + 17:14:12: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:427: shift 1
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; test -w /var/lib/pgsql/9.3/data/'
>>>> > stderr: + 17:14:12: ocf_run:433: output=
>>>> > stderr: + 17:14:12: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:12: ocf_run:435: echo
>>>> > stderr: + 17:14:12: ocf_run:435: output=
>>>> > stderr: + 17:14:12: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:12: ocf_run:437: '[' 1 -a '!' -z '' ']'
>>>> > stderr: + 17:14:12: ocf_run:440: return 0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1797: '[' -n '' -a '!'
>>>> -n '' ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1803: '[' '!' -n '' -a
>>>> -n '' ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1809: is_replication
>>>> > stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a
>>>> sync '!=' slave ']'
>>>> > stderr: + 17:14:12: is_replication:1288: return 0
>>>> > stderr: ++ 17:14:12: pgsql_validate_all:1810: printf '9.3\n9.1'
>>>> > stderr: ++ 17:14:12: pgsql_validate_all:1810: sort -n
>>>> > stderr: ++ 17:14:12: pgsql_validate_all:1810: head -1
>>>> > stderr: + 17:14:12: pgsql_validate_all:1810: '[' 9.1 '!=' 9.1 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1814: '[' '!' -n
>>>> 10.200.200.233 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1820: is_replication
>>>> > stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a
>>>> sync '!=' slave ']'
>>>> > stderr: + 17:14:12: is_replication:1288: return 0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1821: ocf_is_ms
>>>> > stderr: + 17:14:12: ocf_is_ms:504: '[' '!' -z 1 ']'
>>>> > stderr: + 17:14:12: ocf_is_ms:504: '[' 1 -gt 0 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1825: '[' '!' sync =
>>>> sync -a '!' sync = async ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1829: '[' '!' -n 'pg1
>>>> pg2' ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1833: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1834:
>>>> rep_mode_string='include
>>>> '\''/var/lib/pgsql/9.3/tmp//rep_mode.conf'\'' # added by pgsql RA'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1835: '[' sync = sync ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1836: grep -q 'include
>>>> '\''/var/lib/pgsql/9.3/tmp//rep_mode.conf'\'' # added by pgsql RA'
>>>> /var/lib/pgsql/9.3/data//postgresql.conf
>>>> > stderr: + 17:14:12: pgsql_validate_all:1848: mkdir -p
>>>> /var/lib/pgsql/9.3/tmp/
>>>> > stderr: + 17:14:12: pgsql_validate_all:1848: chown postgres
>>>> /var/lib/pgsql/9.3/tmp/
>>>> > stderr: + 17:14:12: pgsql_validate_all:1848: chmod 700
>>>> /var/lib/pgsql/9.3/tmp/
>>>> > stderr: + 17:14:12: pgsql_validate_all:1854: '[' sync = slave ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1861: use_replication_slot
>>>> > stderr: + 17:14:12: use_replication_slot:1294: '[' -n '' ']'
>>>> > stderr: + 17:14:12: use_replication_slot:1298: return 1
>>>> > stderr: + 17:14:12: pgsql_validate_all:1869: return 0
>>>> > stderr: + 17:14:12: 1985: rc=0
>>>> > stderr: + 17:14:12: 1987: '[' promote = validate-all ']'
>>>> > stderr: + 17:14:12: 1989: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: 2002: id -u -n
>>>> > stderr: + 17:14:12: 2002: US=root
>>>> > stderr: + 17:14:12: 2004: '[' root '!=' root -a root '!='
>>>> postgres ']'
>>>> > stderr: + 17:14:12: 2011: '[' -n '' ']'
>>>> > stderr: + 17:14:12: 2016: psql_options='-p 5432 -U postgres
>>>> template1'
>>>> > stderr: + 17:14:12: 2019: '[' -n '' ']'
>>>> > stderr: + 17:14:12: 2022: '[' -n '' ']'
>>>> > stderr: + 17:14:12: 2027: '[' -n 5432 ']'
>>>> > stderr: + 17:14:12: 2028: export PGPORT=5432
>>>> > stderr: + 17:14:12: 2028: PGPORT=5432
>>>> > stderr: + 17:14:12: 2031: '[' -n /usr/lib ']'
>>>> > stderr: + 17:14:12: 2032: '[' -n '' ']'
>>>> > stderr: + 17:14:12: 2035: export LD_LIBRARY_PATH=/usr/lib
>>>> > stderr: + 17:14:12: 2035: LD_LIBRARY_PATH=/usr/lib
>>>> > stderr: + 17:14:12: 2041: case "$1" in
>>>> > stderr: + 17:14:12: 2057: pgsql_promote
>>>> > stderr: + 17:14:12: pgsql_promote:665: local target
>>>> > stderr: + 17:14:12: pgsql_promote:666: local rc
>>>> > stderr: + 17:14:12: pgsql_promote:668: is_replication
>>>> > stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a
>>>> sync '!=' slave ']'
>>>> > stderr: + 17:14:12: is_replication:1288: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:672: rm -f
>>>> '/var/lib/pgsql/9.3/tmp//xlog_note.*'
>>>> > stderr: + 17:14:12: pgsql_promote:674: for target in '$NODE_LIST'
>>>> > stderr: + 17:14:12: pgsql_promote:675: '[' pg1 = pm1 ']'
>>>> > stderr: + 17:14:12: pgsql_promote:676: change_data_status pg1
>>>> DISCONNECT
>>>> > stderr: + 17:14:12: change_data_status:1569: local output
>>>> > stderr: + 17:14:12: change_data_status:1571: node_exist pg1
>>>> > stderr: + 17:14:12: node_exist:1720: print_crm_mon
>>>> > stderr: + 17:14:12: node_exist:1720: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:12: node_exist:1720: grep -q '^node pg1'
>>>> > stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:12: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: + 17:14:12: change_data_status:1572: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:677: change_master_score pg1
>>>> -INFINITY
>>>> > stderr: + 17:14:12: change_master_score:1604: local instance
>>>> > stderr: + 17:14:12: change_master_score:1606: is_node_online pg1
>>>> > stderr: + 17:14:12: is_node_online:1716: print_crm_mon
>>>> > stderr: + 17:14:12: is_node_online:1716: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:12: is_node_online:1716: grep -e '^node pg1 '
>>>> -e '^node pg1:'
>>>> > stderr: + 17:14:12: is_node_online:1716: grep -q -v offline
>>>> > stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:12: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: + 17:14:12: change_master_score:1607: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:674: for target in '$NODE_LIST'
>>>> > stderr: + 17:14:12: pgsql_promote:675: '[' pg2 = pm1 ']'
>>>> > stderr: + 17:14:12: pgsql_promote:676: change_data_status pg2
>>>> DISCONNECT
>>>> > stderr: + 17:14:12: change_data_status:1569: local output
>>>> > stderr: + 17:14:12: change_data_status:1571: node_exist pg2
>>>> > stderr: + 17:14:12: node_exist:1720: print_crm_mon
>>>> > stderr: + 17:14:12: node_exist:1720: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:12: node_exist:1720: grep -q '^node pg2'
>>>> > stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:12: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: + 17:14:12: change_data_status:1572: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:677: change_master_score pg2
>>>> -INFINITY
>>>> > stderr: + 17:14:12: change_master_score:1604: local instance
>>>> > stderr: + 17:14:12: change_master_score:1606: is_node_online pg2
>>>> > stderr: + 17:14:12: is_node_online:1716: grep -q -v offline
>>>> > stderr: + 17:14:12: is_node_online:1716: print_crm_mon
>>>> > stderr: + 17:14:12: is_node_online:1716: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:12: is_node_online:1716: grep -e '^node pg2 '
>>>> -e '^node pg2:'
>>>> > stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:12: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: + 17:14:12: change_master_score:1607: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:680: ocf_log info 'Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>>>> > stderr: + 17:14:12: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:12: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:12: ocf_log:326: shift
>>>> > stderr: + 17:14:12: ocf_log:327: __OCF_MSG='Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>>>> > stderr: + 17:14:12: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:12: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:12: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:12: ocf_log:341: ha_log 'INFO: Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>>>> > stderr: + 17:14:12: ha_log:253: __ha_log 'INFO: Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>>>> > stderr: + 17:14:12: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:12: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:12: __ha_log:188: '[' 'xINFO: Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:12: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:12: __ha_log:192: tty
>>>> > stderr: + 17:14:12: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:12: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:12: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:12: __ha_log:202: echo 'INFO: Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>>>> > stderr: INFO: Creating /var/lib/pgsql/9.3/tmp//PGSQL.lock.
>>>> > stderr: + 17:14:12: __ha_log:204: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:681: touch
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock
>>>> > stderr: + 17:14:12: pgsql_promote:682: show_master_baseline
>>>> > stderr: + 17:14:12: show_master_baseline:1438: local rc
>>>> > stderr: + 17:14:12: show_master_baseline:1439: local location
>>>> > stderr: ++ 17:14:12: show_master_baseline:1441: get_my_location
>>>> > stderr: ++ 17:14:12: get_my_location:1389: local rc
>>>> > stderr: ++ 17:14:12: get_my_location:1390: local output
>>>> > stderr: ++ 17:14:12: get_my_location:1391: local replay_loc
>>>> > stderr: ++ 17:14:12: get_my_location:1392: local receive_loc
>>>> > stderr: ++ 17:14:12: get_my_location:1393: local output1
>>>> > stderr: ++ 17:14:12: get_my_location:1394: local output2
>>>> > stderr: ++ 17:14:12: get_my_location:1395: local log1
>>>> > stderr: ++ 17:14:12: get_my_location:1396: local log2
>>>> > stderr: ++ 17:14:12: get_my_location:1397: local newer_location
>>>> > stderr: +++ 17:14:12: get_my_location:1399: exec_sql ''
>>>> > stderr: +++ 17:14:12: exec_sql:518: local sql=
>>>> > stderr: +++ 17:14:12: exec_sql:519: local output
>>>> > stderr: +++ 17:14:12: exec_sql:520: local rc
>>>> > stderr: ++++ 17:14:12: exec_sql:524: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; /usr/pgsql-9.3/bin/psql -p 5432 -U
>>>> postgres template1 -U postgres -Atc ""'
>>>> > stderr: +++ 17:14:12: exec_sql:524: output=
>>>> > stderr: +++ 17:14:12: exec_sql:525: rc=0
>>>> > stderr: +++ 17:14:12: exec_sql:527: echo
>>>> > stderr: +++ 17:14:12: exec_sql:528: return 0
>>>> > stderr: ++ 17:14:12: get_my_location:1399: output=
>>>> > stderr: ++ 17:14:12: get_my_location:1400: rc=0
>>>> > stderr: ++ 17:14:12: get_my_location:1402: '[' 0 -ne 0 ']'
>>>> > stderr: +++ 17:14:12: get_my_location:1406: cut -d '|' -f 1
>>>> > stderr: +++ 17:14:12: get_my_location:1406: echo
>>>> > stderr: ++ 17:14:12: get_my_location:1406: replay_loc=
>>>> > stderr: +++ 17:14:12: get_my_location:1407: cut -d '|' -f 2
>>>> > stderr: +++ 17:14:12: get_my_location:1407: echo
>>>> > stderr: ++ 17:14:12: get_my_location:1407: receive_loc=
>>>> > stderr: +++ 17:14:12: get_my_location:1409: echo ''
>>>> > stderr: +++ 17:14:12: get_my_location:1409: cut -d / -f 1
>>>> > stderr: ++ 17:14:12: get_my_location:1409: output1=
>>>> > stderr: +++ 17:14:12: get_my_location:1410: echo ''
>>>> > stderr: +++ 17:14:12: get_my_location:1410: cut -d / -f 2
>>>> > stderr: ++ 17:14:12: get_my_location:1410: output2=
>>>> > stderr: +++ 17:14:12: get_my_location:1411: printf '%08s\n'
>>>> > stderr: +++ 17:14:12: get_my_location:1411: sed 's/ /0/g'
>>>> > stderr: ++ 17:14:12: get_my_location:1411: log1=00000000
>>>> > stderr: +++ 17:14:12: get_my_location:1412: printf '%08s\n'
>>>> > stderr: +++ 17:14:12: get_my_location:1412: sed 's/ /0/g'
>>>> > stderr: ++ 17:14:12: get_my_location:1412: log2=00000000
>>>> > stderr: ++ 17:14:12: get_my_location:1413:
>>>> replay_loc=0000000000000000
>>>> > stderr: +++ 17:14:12: get_my_location:1415: cut -d / -f 1
>>>> > stderr: +++ 17:14:12: get_my_location:1415: echo ''
>>>> > stderr: ++ 17:14:12: get_my_location:1415: output1=
>>>> > stderr: +++ 17:14:12: get_my_location:1416: cut -d / -f 2
>>>> > stderr: +++ 17:14:12: get_my_location:1416: echo ''
>>>> > stderr: ++ 17:14:12: get_my_location:1416: output2=
>>>> > stderr: +++ 17:14:12: get_my_location:1417: sed 's/ /0/g'
>>>> > stderr: +++ 17:14:12: get_my_location:1417: printf '%08s\n'
>>>> > stderr: ++ 17:14:12: get_my_location:1417: log1=00000000
>>>> > stderr: +++ 17:14:12: get_my_location:1418: sed 's/ /0/g'
>>>> > stderr: +++ 17:14:12: get_my_location:1418: printf '%08s\n'
>>>> > stderr: ++ 17:14:12: get_my_location:1418: log2=00000000
>>>> > stderr: ++ 17:14:12: get_my_location:1419:
>>>> receive_loc=0000000000000000
>>>> > stderr: +++ 17:14:12: get_my_location:1421: printf
>>>> '0000000000000000\n0000000000000000'
>>>> > stderr: +++ 17:14:12: get_my_location:1421: sort -r
>>>> > stderr: +++ 17:14:12: get_my_location:1421: head -1
>>>> > stderr: ++ 17:14:12: get_my_location:1421:
>>>> newer_location=0000000000000000
>>>> > stderr: ++ 17:14:12: get_my_location:1422: echo 0000000000000000
>>>> > stderr: ++ 17:14:12: get_my_location:1423: return 0
>>>> > stderr: + 17:14:12: show_master_baseline:1441:
>>>> location=0000000000000000
>>>> > stderr: + 17:14:12: show_master_baseline:1442: ocf_log info 'My
>>>> master baseline : 0000000000000000.'
>>>> > stderr: + 17:14:12: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:12: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:12: ocf_log:326: shift
>>>> > stderr: + 17:14:12: ocf_log:327: __OCF_MSG='My master baseline
>>>> : 0000000000000000.'
>>>> > stderr: + 17:14:12: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:12: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:12: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:12: ocf_log:341: ha_log 'INFO: My master
>>>> baseline : 0000000000000000.'
>>>> > stderr: + 17:14:12: ha_log:253: __ha_log 'INFO: My master
>>>> baseline : 0000000000000000.'
>>>> > stderr: + 17:14:12: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:12: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:12: __ha_log:188: '[' 'xINFO: My master
>>>> baseline : 0000000000000000.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:12: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:12: __ha_log:192: tty
>>>> > stderr: + 17:14:12: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:12: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:12: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:12: __ha_log:202: echo 'INFO: My master
>>>> baseline : 0000000000000000.'
>>>> > stderr: INFO: My master baseline : 0000000000000000.
>>>> > stderr: + 17:14:12: __ha_log:204: return 0
>>>> > stderr: + 17:14:12: show_master_baseline:1443: exec_with_retry
>>>> 0 /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-master-baseline
>>>> -v 0000000000000000
>>>> > stderr: + 17:14:12: exec_with_retry:1689: local count=86400
>>>> > stderr: + 17:14:12: exec_with_retry:1690: local output
>>>> > stderr: + 17:14:12: exec_with_retry:1691: local rc
>>>> > stderr: + 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:12: exec_with_retry:1696: shift
>>>> > stderr: + 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1699:
>>>> /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-master-baseline
>>>> -v 0000000000000000
>>>> > stderr: + 17:14:12: exec_with_retry:1699: output=
>>>> > stderr: + 17:14:12: exec_with_retry:1700: rc=0
>>>> > stderr: + 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:12: exec_with_retry:1706: printf ''
>>>> > stderr: + 17:14:12: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:684: ocf_is_true true
>>>> > stderr: + 17:14:12: ocf_is_true:101: case "$1" in
>>>> > stderr: + 17:14:12: ocf_is_true:102: true
>>>> > stderr: + 17:14:12: pgsql_promote:685: ocf_log info 'Restarting
>>>> PostgreSQL instead of promote.'
>>>> > stderr: + 17:14:12: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:12: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:12: ocf_log:326: shift
>>>> > stderr: + 17:14:12: ocf_log:327: __OCF_MSG='Restarting
>>>> PostgreSQL instead of promote.'
>>>> > stderr: + 17:14:12: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:12: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:12: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:12: ocf_log:341: ha_log 'INFO: Restarting
>>>> PostgreSQL instead of promote.'
>>>> > stderr: + 17:14:12: ha_log:253: __ha_log 'INFO: Restarting
>>>> PostgreSQL instead of promote.'
>>>> > stderr: + 17:14:12: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:12: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:12: __ha_log:188: '[' 'xINFO: Restarting
>>>> PostgreSQL instead of promote.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:12: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:12: __ha_log:192: tty
>>>> > stderr: + 17:14:12: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:12: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:12: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:12: __ha_log:202: echo 'INFO: Restarting
>>>> PostgreSQL instead of promote.'
>>>> > stderr: INFO: Restarting PostgreSQL instead of promote.
>>>> > stderr: + 17:14:12: __ha_log:204: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:687: pgsql_real_stop slave
>>>> > stderr: + 17:14:12: pgsql_real_stop:758: local rc
>>>> > stderr: + 17:14:12: pgsql_real_stop:759: local count
>>>> > stderr: + 17:14:12: pgsql_real_stop:760: local stop_escalate
>>>> > stderr: + 17:14:12: pgsql_real_stop:762: ocf_is_true false
>>>> > stderr: + 17:14:12: ocf_is_true:101: case "$1" in
>>>> > stderr: + 17:14:12: ocf_is_true:103: false
>>>> > stderr: + 17:14:12: pgsql_real_stop:766: pgsql_status
>>>> > stderr: + 17:14:12: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: ++ 17:14:12: pgsql_status:868: head -n 1
>>>> /var/lib/pgsql/9.3/data//postmaster.pid
>>>> > stderr: + 17:14:12: pgsql_status:868: PID=6046
>>>> > stderr: + 17:14:12: pgsql_status:869: runasowner 'kill -s 0
>>>> 6046 >/dev/null 2>&1'
>>>> > stderr: + 17:14:12: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:12: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:12: runasowner:469: local var
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; kill -s 0 6046 >/dev/null
>>>> 2>&1'
>>>> > stderr: + 17:14:12: ocf_run:413: local rc
>>>> > stderr: + 17:14:12: ocf_run:414: local output
>>>> > stderr: + 17:14:12: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:12: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:417: local var
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:426: echo -err
>>>> > stderr: ++ 17:14:12: ocf_run:426: sed -e s/-//g
>>>> > stderr: + 17:14:12: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:427: shift 1
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; kill -s 0 6046 >/dev/null 2>&1'
>>>> > stderr: + 17:14:12: ocf_run:433: output=
>>>> > stderr: + 17:14:12: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:12: ocf_run:435: echo
>>>> > stderr: + 17:14:12: ocf_run:435: output=
>>>> > stderr: + 17:14:12: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:12: ocf_run:437: '[' 1 -a '!' -z '' ']'
>>>> > stderr: + 17:14:12: ocf_run:440: return 0
>>>> > stderr: + 17:14:12: pgsql_status:870: return 0
>>>> > stderr: + 17:14:12: pgsql_real_stop:772: stop_escalate=30
>>>> > stderr: + 17:14:12: pgsql_real_stop:773: '[' slave = slave ']'
>>>> > stderr: + 17:14:12: pgsql_real_stop:774: stop_escalate=30
>>>> > stderr: + 17:14:12: pgsql_real_stop:778: '[' 30 -gt 0 ']'
>>>> > stderr: + 17:14:12: pgsql_real_stop:779: runasowner
>>>> '/usr/pgsql-9.3/bin/pg_ctl -D /var/lib/pgsql/9.3/data/ stop -m fast'
>>>> > stderr: + 17:14:12: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:12: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:12: runasowner:469: local var
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; /usr/pgsql-9.3/bin/pg_ctl
>>>> -D /var/lib/pgsql/9.3/data/ stop -m fast'
>>>> > stderr: + 17:14:12: ocf_run:413: local rc
>>>> > stderr: + 17:14:12: ocf_run:414: local output
>>>> > stderr: + 17:14:12: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:12: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:417: local var
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:426: echo -err
>>>> > stderr: ++ 17:14:12: ocf_run:426: sed -e s/-//g
>>>> > stderr: + 17:14:12: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:427: shift 1
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; /usr/pgsql-9.3/bin/pg_ctl -D
>>>> /var/lib/pgsql/9.3/data/ stop -m fast'
>>>> > stderr: + 17:14:13: ocf_run:433: output='waiting for server to
>>>> shut down.... done
>>>> > stderr: server stopped'
>>>> > stderr: + 17:14:13: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:13: ocf_run:435: echo waiting for server to
>>>> shut down.... done server stopped
>>>> > stderr: + 17:14:13: ocf_run:435: output='waiting for server to
>>>> shut down.... done server stopped'
>>>> > stderr: + 17:14:13: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:13: ocf_run:437: '[' 1 -a '!' -z 'waiting for
>>>> server to shut down.... done server stopped' ']'
>>>> > stderr: + 17:14:13: ocf_run:438: ocf_log info 'waiting for
>>>> server to shut down.... done server stopped'
>>>> > stderr: + 17:14:13: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:13: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:13: ocf_log:326: shift
>>>> > stderr: + 17:14:13: ocf_log:327: __OCF_MSG='waiting for server
>>>> to shut down.... done server stopped'
>>>> > stderr: + 17:14:13: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:13: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:13: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:13: ocf_log:341: ha_log 'INFO: waiting for
>>>> server to shut down.... done server stopped'
>>>> > stderr: + 17:14:13: ha_log:253: __ha_log 'INFO: waiting for
>>>> server to shut down.... done server stopped'
>>>> > stderr: + 17:14:13: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:13: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:13: __ha_log:188: '[' 'xINFO: waiting for
>>>> server to shut down.... done server stopped' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:13: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:13: __ha_log:192: tty
>>>> > stderr: + 17:14:13: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:13: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:13: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:13: __ha_log:202: echo 'INFO: waiting for
>>>> server to shut down.... done server stopped'
>>>> > stderr: INFO: waiting for server to shut down.... done server
>>>> stopped
>>>> > stderr: + 17:14:13: __ha_log:204: return 0
>>>> > stderr: + 17:14:13: ocf_run:440: return 0
>>>> > stderr: + 17:14:13: pgsql_real_stop:783: count=0
>>>> > stderr: + 17:14:13: pgsql_real_stop:784: '[' 0 -lt 30 ']'
>>>> > stderr: + 17:14:13: pgsql_real_stop:786: pgsql_status
>>>> > stderr: + 17:14:13: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: + 17:14:13: pgsql_status:874: false
>>>> > stderr: + 17:14:13: pgsql_real_stop:789: break
>>>> > stderr: + 17:14:13: pgsql_real_stop:795: pgsql_status
>>>> > stderr: + 17:14:13: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: + 17:14:13: pgsql_status:874: false
>>>> > stderr: + 17:14:13: pgsql_real_stop:802: :
>>>> > stderr: + 17:14:13: pgsql_real_stop:804: pgsql_real_monitor
>>>> > stderr: + 17:14:13: pgsql_real_monitor:905: local loglevel
>>>> > stderr: + 17:14:13: pgsql_real_monitor:906: local rc
>>>> > stderr: + 17:14:13: pgsql_real_monitor:907: local output
>>>> > stderr: + 17:14:13: pgsql_real_monitor:910: loglevel=err
>>>> > stderr: + 17:14:13: pgsql_real_monitor:912: pgsql_status
>>>> > stderr: + 17:14:13: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: + 17:14:13: pgsql_status:874: false
>>>> > stderr: + 17:14:13: pgsql_real_monitor:914: ocf_log info
>>>> 'PostgreSQL is down'
>>>> > stderr: + 17:14:13: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:13: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:13: ocf_log:326: shift
>>>> > stderr: + 17:14:13: ocf_log:327: __OCF_MSG='PostgreSQL is down'
>>>> > stderr: + 17:14:13: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:13: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:13: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:13: ocf_log:341: ha_log 'INFO: PostgreSQL is down'
>>>> > stderr: + 17:14:13: ha_log:253: __ha_log 'INFO: PostgreSQL is
>>>> down'
>>>> > stderr: + 17:14:13: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:13: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:13: __ha_log:188: '[' 'xINFO: PostgreSQL is
>>>> down' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:13: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:13: __ha_log:192: tty
>>>> > stderr: + 17:14:13: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:13: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:13: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:13: __ha_log:202: echo 'INFO: PostgreSQL is down'
>>>> > stderr: INFO: PostgreSQL is down
>>>> > stderr: + 17:14:13: __ha_log:204: return 0
>>>> > stderr: + 17:14:13: pgsql_real_monitor:915: return 7
>>>> > stderr: + 17:14:13: pgsql_real_stop:805: rc=7
>>>> > stderr: + 17:14:13: pgsql_real_stop:806: '[' 7 -eq 7 ']'
>>>> > stderr: + 17:14:13: pgsql_real_stop:808: break
>>>> > stderr: + 17:14:13: pgsql_real_stop:815: rm -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid
>>>> > stderr: + 17:14:13: pgsql_real_stop:817: '[' slave = master -a
>>>> '' = ' ' ']'
>>>> > stderr: + 17:14:13: pgsql_real_stop:821: return 0
>>>> > stderr: + 17:14:13: pgsql_promote:688: rm -f
>>>> /var/lib/pgsql/9.3/data//recovery.conf
>>>> > stderr: + 17:14:13: pgsql_promote:689: pgsql_real_start
>>>> > stderr: + 17:14:13: pgsql_real_start:534: local pgctl_options
>>>> > stderr: + 17:14:13: pgsql_real_start:535: local postgres_options
>>>> > stderr: + 17:14:13: pgsql_real_start:536: local rc
>>>> > stderr: + 17:14:13: pgsql_real_start:538: pgsql_status
>>>> > stderr: + 17:14:13: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: + 17:14:13: pgsql_status:874: false
>>>> > stderr: + 17:14:13: pgsql_real_start:548: rm -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid
>>>> > stderr: + 17:14:13: pgsql_real_start:551: '[' -f
>>>> /var/lib/pgsql/9.3/data//backup_label ']'
>>>> > stderr: + 17:14:13: pgsql_real_start:557: check_log_file /dev/null
>>>> > stderr: + 17:14:13: check_log_file:1878: '[' '!' -f /dev/null ']'
>>>> > stderr: + 17:14:13: check_log_file:1880: touch /dev/null
>>>> > stderr: ++ 17:14:13: check_log_file:1881: getent passwd postgres
>>>> > stderr: ++ 17:14:13: check_log_file:1881: cut -d : -f 4
>>>> > stderr: + 17:14:13: check_log_file:1881: chown postgres:26
>>>> /dev/null
>>>> > stderr: + 17:14:13: check_log_file:1885: runasowner 'test -w
>>>> /dev/null'
>>>> > stderr: + 17:14:13: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:13: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:13: runasowner:469: local var
>>>> > stderr: + 17:14:13: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:13: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:13: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:13: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:13: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; test -w /dev/null'
>>>> > stderr: + 17:14:13: ocf_run:413: local rc
>>>> > stderr: + 17:14:13: ocf_run:414: local output
>>>> > stderr: + 17:14:13: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:13: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:13: ocf_run:417: local var
>>>> > stderr: + 17:14:13: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:13: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:13: ocf_run:426: echo -err
>>>> > stderr: ++ 17:14:13: ocf_run:426: sed -e s/-//g
>>>> > stderr: + 17:14:13: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:13: ocf_run:427: shift 1
>>>> > stderr: + 17:14:13: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:13: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:13: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; test -w /dev/null'
>>>> > stderr: + 17:14:13: ocf_run:433: output=
>>>> > stderr: + 17:14:13: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:13: ocf_run:435: echo
>>>> > stderr: + 17:14:13: ocf_run:435: output=
>>>> > stderr: + 17:14:13: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:13: ocf_run:437: '[' 1 -a '!' -z '' ']'
>>>> > stderr: + 17:14:13: ocf_run:440: return 0
>>>> > stderr: + 17:14:13: check_log_file:1890: return 0
>>>> > stderr: + 17:14:13: pgsql_real_start:564: '[' -n '' ']'
>>>> > stderr: + 17:14:13: pgsql_real_start:569: '[' sync = slave ']'
>>>> > stderr: + 17:14:13: pgsql_real_start:575: pgctl_options=' -D
>>>> /var/lib/pgsql/9.3/data/ -l /dev/null'
>>>> > stderr: + 17:14:13: pgsql_real_start:578: postgres_options='-c
>>>> config_file=/var/lib/pgsql/9.3/data//postgresql.conf'
>>>> > stderr: + 17:14:13: pgsql_real_start:580: '[' -n '' ']'
>>>> > stderr: + 17:14:13: pgsql_real_start:583: '[' -n '' ']'
>>>> > stderr: + 17:14:13: pgsql_real_start:588: pgctl_options=' -D
>>>> /var/lib/pgsql/9.3/data/ -l /dev/null -o '\''-c
>>>> config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\'''
>>>> > stderr: + 17:14:13: pgsql_real_start:591: runasowner 'unset
>>>> PGUSER; unset PGPASSWORD; /usr/pgsql-9.3/bin/pg_ctl -D
>>>> /var/lib/pgsql/9.3/data/ -l /dev/null -o '\''-c
>>>> config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\'' start'
>>>> > stderr: + 17:14:13: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:13: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:13: runasowner:469: local var
>>>> > stderr: + 17:14:13: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:13: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:13: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:13: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:13: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; unset PGUSER; unset
>>>> PGPASSWORD; /usr/pgsql-9.3/bin/pg_ctl -D /var/lib/pgsql/9.3/data/
>>>> -l /dev/null -o '\''-c
>>>> config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\'' start'
>>>> > stderr: + 17:14:13: ocf_run:413: local rc
>>>> > stderr: + 17:14:13: ocf_run:414: local output
>>>> > stderr: + 17:14:13: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:13: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:13: ocf_run:417: local var
>>>> > stderr: + 17:14:13: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:13: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:13: ocf_run:426: sed -e s/-//g
>>>> > stderr: ++ 17:14:13: ocf_run:426: echo -err
>>>> > stderr: + 17:14:13: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:13: ocf_run:427: shift 1
>>>> > stderr: + 17:14:13: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:13: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:13: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; unset PGUSER; unset PGPASSWORD;
>>>> /usr/pgsql-9.3/bin/pg_ctl -D /var/lib/pgsql/9.3/data/ -l /dev/null
>>>> -o '\''-c config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\''
>>>> start'
>>>> > stderr: + 17:14:13: ocf_run:433: output='server starting'
>>>> > stderr: + 17:14:13: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:13: ocf_run:435: echo server starting
>>>> > stderr: + 17:14:13: ocf_run:435: output='server starting'
>>>> > stderr: + 17:14:13: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:13: ocf_run:437: '[' 1 -a '!' -z 'server
>>>> starting' ']'
>>>> > stderr: + 17:14:13: ocf_run:438: ocf_log info 'server starting'
>>>> > stderr: + 17:14:13: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:13: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:13: ocf_log:326: shift
>>>> > stderr: + 17:14:13: ocf_log:327: __OCF_MSG='server starting'
>>>> > stderr: + 17:14:13: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:13: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: server starting'
>>>> > stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: server starting'
>>>> > stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:14: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: server starting'
>>>> = x--ignore-stderr ']'
>>>> > stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:14: __ha_log:192: tty
>>>> > stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:14: __ha_log:202: echo 'INFO: server starting'
>>>> > stderr: INFO: server starting
>>>> > stderr: + 17:14:14: __ha_log:204: return 0
>>>> > stderr: + 17:14:14: ocf_run:440: return 0
>>>> > stderr: + 17:14:14: pgsql_real_start:593: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:14: pgsql_real_start:595: ocf_log info
>>>> 'PostgreSQL start command sent.'
>>>> > stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:14: ocf_log:326: shift
>>>> > stderr: + 17:14:14: ocf_log:327: __OCF_MSG='PostgreSQL start
>>>> command sent.'
>>>> > stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:14: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: PostgreSQL start
>>>> command sent.'
>>>> > stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: PostgreSQL
>>>> start command sent.'
>>>> > stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:14: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: PostgreSQL start
>>>> command sent.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:14: __ha_log:192: tty
>>>> > stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:14: __ha_log:202: echo 'INFO: PostgreSQL start
>>>> command sent.'
>>>> > stderr: INFO: PostgreSQL start command sent.
>>>> > stderr: + 17:14:14: __ha_log:204: return 0
>>>> > stderr: + 17:14:14: pgsql_real_start:601: :
>>>> > stderr: + 17:14:14: pgsql_real_start:603: pgsql_real_monitor warn
>>>> > stderr: + 17:14:14: pgsql_real_monitor:905: local loglevel
>>>> > stderr: + 17:14:14: pgsql_real_monitor:906: local rc
>>>> > stderr: + 17:14:14: pgsql_real_monitor:907: local output
>>>> > stderr: + 17:14:14: pgsql_real_monitor:910: loglevel=warn
>>>> > stderr: + 17:14:14: pgsql_real_monitor:912: pgsql_status
>>>> > stderr: + 17:14:14: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: ++ 17:14:14: pgsql_status:868: head -n 1
>>>> /var/lib/pgsql/9.3/data//postmaster.pid
>>>> > stderr: + 17:14:14: pgsql_status:868: PID=9556
>>>> > stderr: + 17:14:14: pgsql_status:869: runasowner 'kill -s 0
>>>> 9556 >/dev/null 2>&1'
>>>> > stderr: + 17:14:14: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:14: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:14: runasowner:469: local var
>>>> > stderr: + 17:14:14: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:14: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:14: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:14: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:14: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; kill -s 0 9556 >/dev/null
>>>> 2>&1'
>>>> > stderr: + 17:14:14: ocf_run:413: local rc
>>>> > stderr: + 17:14:14: ocf_run:414: local output
>>>> > stderr: + 17:14:14: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:14: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:14: ocf_run:417: local var
>>>> > stderr: + 17:14:14: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:14: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:14: ocf_run:426: echo -err
>>>> > stderr: ++ 17:14:14: ocf_run:426: sed -e s/-//g
>>>> > stderr: + 17:14:14: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:14: ocf_run:427: shift 1
>>>> > stderr: + 17:14:14: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:14: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:14: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; kill -s 0 9556 >/dev/null 2>&1'
>>>> > stderr: + 17:14:14: ocf_run:433: output=
>>>> > stderr: + 17:14:14: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:14: ocf_run:435: echo
>>>> > stderr: + 17:14:14: ocf_run:435: output=
>>>> > stderr: + 17:14:14: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:14: ocf_run:437: '[' 1 -a '!' -z '' ']'
>>>> > stderr: + 17:14:14: ocf_run:440: return 0
>>>> > stderr: + 17:14:14: pgsql_status:870: return 0
>>>> > stderr: + 17:14:14: pgsql_real_monitor:918: is_replication
>>>> > stderr: + 17:14:14: is_replication:1287: '[' sync '!=' none -a
>>>> sync '!=' slave ']'
>>>> > stderr: + 17:14:14: is_replication:1288: return 0
>>>> > stderr: ++ 17:14:14: pgsql_real_monitor:920: exec_sql 'select
>>>> pg_is_in_recovery()'
>>>> > stderr: ++ 17:14:14: exec_sql:518: local 'sql=select
>>>> pg_is_in_recovery()'
>>>> > stderr: ++ 17:14:14: exec_sql:519: local output
>>>> > stderr: ++ 17:14:14: exec_sql:520: local rc
>>>> > stderr: +++ 17:14:14: exec_sql:524: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; /usr/pgsql-9.3/bin/psql -p 5432 -U
>>>> postgres template1 -U postgres -Atc "select
>>>> pg_is_in_recovery()"'
>>>> > stderr: ++ 17:14:14: exec_sql:524: output=f
>>>> > stderr: ++ 17:14:14: exec_sql:525: rc=0
>>>> > stderr: ++ 17:14:14: exec_sql:527: echo f
>>>> > stderr: ++ 17:14:14: exec_sql:528: return 0
>>>> > stderr: + 17:14:14: pgsql_real_monitor:920: output=f
>>>> > stderr: + 17:14:14: pgsql_real_monitor:921: rc=0
>>>> > stderr: + 17:14:14: pgsql_real_monitor:923: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:14: pgsql_real_monitor:928: case "$output" in
>>>> > stderr: + 17:14:14: pgsql_real_monitor:929: ocf_log debug
>>>> 'PostgreSQL is running as a primary.'
>>>> > stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=debug
>>>> > stderr: + 17:14:14: ocf_log:326: shift
>>>> > stderr: + 17:14:14: ocf_log:327: __OCF_MSG='PostgreSQL is
>>>> running as a primary.'
>>>> > stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:14: ocf_log:334: __OCF_PRIO=DEBUG
>>>> > stderr: + 17:14:14: ocf_log:338: '[' DEBUG = DEBUG ']'
>>>> > stderr: + 17:14:14: ocf_log:339: ha_debug 'DEBUG: PostgreSQL is
>>>> running as a primary.'
>>>> > stderr: + 17:14:14: ha_debug:258: '[' x = x0 ']'
>>>> > stderr: + 17:14:14: ha_debug:261: tty
>>>> > stderr: + 17:14:14: ha_debug:262: '[' '' ']'
>>>> > stderr: + 17:14:14: ha_debug:265: echo 'DEBUG: PostgreSQL is
>>>> running as a primary.'
>>>> > stderr: DEBUG: PostgreSQL is running as a primary.
>>>> > stderr: + 17:14:14: ha_debug:267: return 0
>>>> > stderr: + 17:14:14: pgsql_real_monitor:930: '[' 'select now();'
>>>> = 'select now();' ']'
>>>> > stderr: + 17:14:14: pgsql_real_monitor:931: return 8
>>>> > stderr: + 17:14:14: pgsql_real_start:604: rc=8
>>>> > stderr: + 17:14:14: pgsql_real_start:605: '[' 8 -eq 0 -o 8 -eq
>>>> 8 ']'
>>>> > stderr: + 17:14:14: pgsql_real_start:606: break
>>>> > stderr: + 17:14:14: pgsql_real_start:614: use_replication_slot
>>>> > stderr: + 17:14:14: use_replication_slot:1294: '[' -n '' ']'
>>>> > stderr: + 17:14:14: use_replication_slot:1298: return 1
>>>> > stderr: + 17:14:14: pgsql_real_start:622: ocf_log info
>>>> 'PostgreSQL is started.'
>>>> > stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:14: ocf_log:326: shift
>>>> > stderr: + 17:14:14: ocf_log:327: __OCF_MSG='PostgreSQL is
>>>> started.'
>>>> > stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:14: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: PostgreSQL is
>>>> started.'
>>>> > stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: PostgreSQL is
>>>> started.'
>>>> > stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:14: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: PostgreSQL is
>>>> started.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:14: __ha_log:192: tty
>>>> > stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:14: __ha_log:202: echo 'INFO: PostgreSQL is
>>>> started.'
>>>> > stderr: INFO: PostgreSQL is started.
>>>> > stderr: + 17:14:14: __ha_log:204: return 0
>>>> > stderr: + 17:14:14: pgsql_real_start:623: return 8
>>>> > stderr: + 17:14:14: pgsql_promote:690: rc=8
>>>> > stderr: + 17:14:14: pgsql_promote:691: '[' 8 -ne 8 ']'
>>>> > stderr: + 17:14:14: pgsql_promote:723: change_data_status pm1
>>>> LATEST
>>>> > stderr: + 17:14:14: change_data_status:1569: local output
>>>> > stderr: + 17:14:14: change_data_status:1571: node_exist pm1
>>>> > stderr: + 17:14:14: node_exist:1720: print_crm_mon
>>>> > stderr: + 17:14:14: node_exist:1720: grep -q '^node pm1'
>>>> > stderr: + 17:14:14: node_exist:1720: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:14: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:14: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:14: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:14: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:14: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:14: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:14: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:14: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:14: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: + 17:14:14: change_data_status:1575: :
>>>> > stderr: ++ 17:14:14: change_data_status:1577:
>>>> /usr/sbin/crm_attribute -l forever -N pm1 -n pgsql-data-status -G -q
>>>> > stderr: + 17:14:14: change_data_status:1577: output=LATEST
>>>> > stderr: + 17:14:14: change_data_status:1578: '[' LATEST '!='
>>>> LATEST ']'
>>>> > stderr: + 17:14:14: change_data_status:1582: break
>>>> > stderr: + 17:14:14: change_data_status:1585: return 0
>>>> > stderr: + 17:14:14: pgsql_promote:724: exec_with_retry 0
>>>> /usr/sbin/crm_master -l reboot -v 1000
>>>> > stderr: + 17:14:14: exec_with_retry:1689: local count=86400
>>>> > stderr: + 17:14:14: exec_with_retry:1690: local output
>>>> > stderr: + 17:14:14: exec_with_retry:1691: local rc
>>>> > stderr: + 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:14: exec_with_retry:1696: shift
>>>> > stderr: + 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1699: /usr/sbin/crm_master
>>>> -l reboot -v 1000
>>>> > stderr: + 17:14:14: exec_with_retry:1699: output=
>>>> > stderr: + 17:14:14: exec_with_retry:1700: rc=0
>>>> > stderr: + 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:14: exec_with_retry:1706: printf ''
>>>> > stderr: + 17:14:14: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:14: pgsql_promote:725: change_pgsql_status pm1 PRI
>>>> > stderr: + 17:14:14: change_pgsql_status:1543: local output
>>>> > stderr: + 17:14:14: change_pgsql_status:1545: is_node_online pm1
>>>> > stderr: + 17:14:14: is_node_online:1716: print_crm_mon
>>>> > stderr: + 17:14:14: is_node_online:1716: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:14: is_node_online:1716: grep -e '^node pm1 '
>>>> -e '^node pm1:'
>>>> > stderr: + 17:14:14: is_node_online:1716: grep -q -v offline
>>>> > stderr: + 17:14:14: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:14: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:14: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:14: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:14: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:14: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:14: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:14: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:14: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: ++ 17:14:14: change_pgsql_status:1549:
>>>> /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-status -G -q
>>>> > stderr: + 17:14:14: change_pgsql_status:1549: output=HS:alone
>>>> > stderr: + 17:14:14: change_pgsql_status:1550: '[' HS:alone '!='
>>>> PRI ']'
>>>> > stderr: + 17:14:14: change_pgsql_status:1554: '[' HS:alone =
>>>> STOP -o HS:alone = UNKNOWN ']'
>>>> > stderr: + 17:14:14: change_pgsql_status:1560: ocf_log info
>>>> 'Changing pgsql-status on pm1 : HS:alone->PRI.'
>>>> > stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:14: ocf_log:326: shift
>>>> > stderr: + 17:14:14: ocf_log:327: __OCF_MSG='Changing
>>>> pgsql-status on pm1 : HS:alone->PRI.'
>>>> > stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:14: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: Changing
>>>> pgsql-status on pm1 : HS:alone->PRI.'
>>>> > stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: Changing
>>>> pgsql-status on pm1 : HS:alone->PRI.'
>>>> > stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:14: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: Changing
>>>> pgsql-status on pm1 : HS:alone->PRI.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:14: __ha_log:192: tty
>>>> > stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:14: __ha_log:202: echo 'INFO: Changing
>>>> pgsql-status on pm1 : HS:alone->PRI.'
>>>> > stderr: INFO: Changing pgsql-status on pm1 : HS:alone->PRI.
>>>> > stderr: + 17:14:14: __ha_log:204: return 0
>>>> > stderr: + 17:14:14: change_pgsql_status:1561: exec_with_retry 0
>>>> /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-status -v PRI
>>>> > stderr: + 17:14:14: exec_with_retry:1689: local count=86400
>>>> > stderr: + 17:14:14: exec_with_retry:1690: local output
>>>> > stderr: + 17:14:14: exec_with_retry:1691: local rc
>>>> > stderr: + 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:14: exec_with_retry:1696: shift
>>>> > stderr: + 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1699:
>>>> /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-status -v PRI
>>>> > stderr: + 17:14:14: exec_with_retry:1699: output=
>>>> > stderr: + 17:14:14: exec_with_retry:1700: rc=0
>>>> > stderr: + 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:14: exec_with_retry:1706: printf ''
>>>> > stderr: + 17:14:14: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:14: change_pgsql_status:1563: return 0
>>>> > stderr: + 17:14:14: pgsql_promote:726: return 0
>>>> > stderr: + 17:14:14: 2058: exit 0
>>>>
>>>>
>>>> Thanks in advance!
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list: Users at clusterlabs.org
>>>> http://clusterlabs.org/mailman/listinfo/users
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started:
>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://bugs.clusterlabs.org
>>>
>>
>>
>> _______________________________________________
>> Users mailing list: Users at clusterlabs.org
>> http://clusterlabs.org/mailman/listinfo/users
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>
> --
> Nikolay Popov
> n.popov at postgrespro.ru
> Postgres Professional:http://www.postgrespro.com
> The Russian Postgres Company
>
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20151012/4e43f91e/attachment.htm>
More information about the Users
mailing list