[ClusterLabs] pgsql Master/Slave set has no promote (all nodes are slaves)
Andrey Rubik
tirnotaure at gmail.com
Mon Oct 12 11:43:51 EDT 2015
This is debug log after start cluster:
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: crm_log_init: Changed
active directory to /var/lib/pacemaker/cores/root
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: main: Checking for
old instances of pacemakerd
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: crm_ipc_connect:
Could not establish pacemakerd connection: Connection refused (111)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: get_cluster_type:
Testing with Corosync
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31928-17-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31928-17-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31928-17-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: get_cluster_type:
Detected an active 'corosync' cluster
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: mcp_read_config:
Reading configure for stack: corosync
Oct 12 18:41:05 [31928] pm1 pacemakerd: notice: mcp_read_config:
Configured corosync to accept connections from group 189: OK (1)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31928-16-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31928-16-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31928-16-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: notice: main: Starting
Pacemaker 1.1.13 (Build: a14efad): generated-manpages agent-manpages
ascii-docs publican-docs ncurses libqb-logging libqb-ipc upstart systemd
nagios corosync-native atomic-attrd acls
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: main: Maximum core
file size is: 18446744073709551615
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: qb_ipcs_us_publish:
server name: pacemakerd
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
cluster_connect_cfg: Our nodeid: 1
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31928-18-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31928-18-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31928-18-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: crm_get_peer: Created
entry a6299078-f254-4732-a62a-4e6406d8719f/0x2320290 for node pm1/1 (1
total)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: crm_get_peer: Node 1
is now known as pm1
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31928-18-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31928-18-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31928-18-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: crm_get_peer: Node 1
has uuid 1
Oct 12 18:41:05 [31928] pm1 pacemakerd: info:
crm_update_peer_proc: cluster_connect_cpg: Node pm1[1] - corosync-cpg
is now online
Oct 12 18:41:05 [31928] pm1 pacemakerd: notice:
crm_update_peer_state: crm_update_peer_proc: Node pm1[1] - state is
now member (was (null))
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
cluster_connect_quorum: Configuring Pacemaker to obtain quorum from
Corosync
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: notice:
cluster_connect_quorum: Quorum acquired
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: crm_get_peer: Created
entry f37cc001-0b8c-491b-98a0-8dd91055ecb4/0x2321720 for node pm2/2 (2
total)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: crm_get_peer: Node 2
is now known as pm2
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: crm_get_peer: Node 2
has uuid 2
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31928-19-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31928-19-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31928-19-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31928-19-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31928-19-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31928-19-header
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: start_child: Using
uid=189 and group=189 for process cib
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: start_child: Forked
child 31929 for process cib
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm1 now has process list:
00000000000000000000000004000100 (was 00000000000000000000000004000000)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: start_child: Forked
child 31930 for process stonith-ng
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm1 now has process list:
00000000000000000000000004100100 (was 00000000000000000000000004000100)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: start_child: Forked
child 31931 for process lrmd
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm1 now has process list:
00000000000000000000000004100110 (was 00000000000000000000000004100100)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: start_child: Using
uid=189 and group=189 for process attrd
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: start_child: Forked
child 31932 for process attrd
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm1 now has process list:
00000000000000000000000004101110 (was 00000000000000000000000004100110)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: start_child: Using
uid=189 and group=189 for process pengine
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: start_child: Forked
child 31933 for process pengine
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm1 now has process list:
00000000000000000000000004111110 (was 00000000000000000000000004101110)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: start_child: Using
uid=189 and group=189 for process crmd
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: start_child: Forked
child 31934 for process crmd
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm1 now has process list:
00000000000000000000000004111310 (was 00000000000000000000000004111110)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: main: Starting mainloop
Oct 12 18:41:05 [31928] pm1 pacemakerd: info:
pcmk_quorum_notification: Membership 172: quorum retained (2)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
pcmk_quorum_notification: Member[0] 1
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
pcmk_quorum_notification: Member[1] 2
Oct 12 18:41:05 [31928] pm1 pacemakerd: notice:
crm_update_peer_state: pcmk_quorum_notification: Node pm2[2] - state
is now member (was (null))
Oct 12 18:41:05 [31928] pm1 pacemakerd: info:
pcmk_cpg_membership: Node 1 joined group pacemakerd (counter=0.0)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info:
pcmk_cpg_membership: Node 1 still member of group pacemakerd
(peer=pm1, counter=0.0)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: mcp_cpg_deliver:
Ignoring process list sent by peer for local node
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: mcp_cpg_deliver:
Ignoring process list sent by peer for local node
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: mcp_cpg_deliver:
Ignoring process list sent by peer for local node
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: mcp_cpg_deliver:
Ignoring process list sent by peer for local node
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: mcp_cpg_deliver:
Ignoring process list sent by peer for local node
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: mcp_cpg_deliver:
Ignoring process list sent by peer for local node
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: mcp_cpg_deliver:
Ignoring process list sent by peer for local node
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: child_waitpid:
wait(31929) = 0: Success (0)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: child_waitpid:
wait(31930) = 0: Success (0)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: child_waitpid:
wait(31931) = 0: Success (0)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: child_waitpid:
wait(31932) = 0: Success (0)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: child_waitpid:
wait(31933) = 0: Success (0)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug: child_waitpid:
wait(31934) = 0: Success (0)
Oct 12 18:41:05 [31929] pm1 cib: info: crm_log_init: Changed
active directory to /var/lib/pacemaker/cores/hacluster
Oct 12 18:41:05 [31929] pm1 cib: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:05 [31929] pm1 cib: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:05 [31929] pm1 cib: info: retrieveCib: Reading
cluster configuration file /var/lib/pacemaker/cib/cib.xml (digest:
/var/lib/pacemaker/cib/cib.xml.sig)
Oct 12 18:41:05 [31929] pm1 cib: info:
validate_with_relaxng: Creating RNG parser context
Oct 12 18:41:05 [31928] pm1 pacemakerd: info:
pcmk_cpg_membership: Node 2 joined group pacemakerd (counter=1.0)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info:
pcmk_cpg_membership: Node 1 still member of group pacemakerd
(peer=pm1, counter=1.0)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info:
pcmk_cpg_membership: Node 2 still member of group pacemakerd
(peer=pm2, counter=1.1)
Oct 12 18:41:05 [31928] pm1 pacemakerd: info:
crm_update_peer_proc: pcmk_cpg_membership: Node pm2[2] - corosync-cpg
is now online
Oct 12 18:41:05 [31928] pm1 pacemakerd: info: mcp_cpg_deliver:
Ignoring process list sent by peer for local node
Oct 12 18:41:05 [31932] pm1 attrd: info: crm_log_init: Changed
active directory to /var/lib/pacemaker/cores/hacluster
Oct 12 18:41:05 [31932] pm1 attrd: info: main: Starting up
Oct 12 18:41:05 [31932] pm1 attrd: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:05 [31932] pm1 attrd: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:05 [31932] pm1 attrd: notice:
crm_cluster_connect: Connecting to cluster infrastructure: corosync
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm2 now has process list:
00000000000000000000000004000100 (was 00000000000000000000000004000000)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm2 now has process list:
00000000000000000000000004100100 (was 00000000000000000000000004000100)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm2 now has process list:
00000000000000000000000004100110 (was 00000000000000000000000004100100)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm2 now has process list:
00000000000000000000000004101110 (was 00000000000000000000000004100110)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm2 now has process list:
00000000000000000000000004111110 (was 00000000000000000000000004101110)
Oct 12 18:41:05 [31928] pm1 pacemakerd: debug:
update_node_processes: Node pm2 now has process list:
00000000000000000000000004111310 (was 00000000000000000000000004111110)
Oct 12 18:41:05 [31933] pm1 pengine: info: crm_log_init: Changed
active directory to /var/lib/pacemaker/cores/hacluster
Oct 12 18:41:05 [31933] pm1 pengine: debug: main: Init server comms
Oct 12 18:41:05 [31933] pm1 pengine: info: qb_ipcs_us_publish:
server name: pengine
Oct 12 18:41:05 [31933] pm1 pengine: info: main: Starting pengine
Oct 12 18:41:05 [31934] pm1 crmd: info: crm_log_init: Changed
active directory to /var/lib/pacemaker/cores/hacluster
Oct 12 18:41:05 [31934] pm1 crmd: notice: main: CRM Git
Version: a14efad
Oct 12 18:41:05 [31934] pm1 crmd: debug: crmd_init: Starting crmd
Oct 12 18:41:05 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_STARTUP: [ state=S_STARTING cause=C_STARTUP origin=crmd_init ]
Oct 12 18:41:05 [31934] pm1 crmd: info: do_log: FSA: Input
I_STARTUP from crmd_init() received in state S_STARTING
Oct 12 18:41:05 [31934] pm1 crmd: debug: do_startup:
Registering Signal Handlers
Oct 12 18:41:05 [31934] pm1 crmd: debug: do_startup: Creating
CIB and LRM objects
Oct 12 18:41:05 [31934] pm1 crmd: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:05 [31934] pm1 crmd: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:05 [31934] pm1 crmd: debug: crm_ipc_connect:
Could not establish cib_shm connection: Connection refused (111)
Oct 12 18:41:05 [31934] pm1 crmd: debug:
cib_native_signon_raw: Connection unsuccessful (0 (nil))
Oct 12 18:41:05 [31934] pm1 crmd: debug:
cib_native_signon_raw: Connection to CIB failed: Transport endpoint
is not connected
Oct 12 18:41:05 [31934] pm1 crmd: debug: cib_native_signoff:
Signing out of the CIB Service
Oct 12 18:41:05 [31931] pm1 lrmd: info: crm_log_init: Changed
active directory to /var/lib/pacemaker/cores/root
Oct 12 18:41:05 [31931] pm1 lrmd: info: qb_ipcs_us_publish:
server name: lrmd
Oct 12 18:41:05 [31931] pm1 lrmd: info: main: Starting
Oct 12 18:41:05 [31930] pm1 stonith-ng: info: crm_log_init: Changed
active directory to /var/lib/pacemaker/cores/root
Oct 12 18:41:05 [31930] pm1 stonith-ng: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:05 [31930] pm1 stonith-ng: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:05 [31930] pm1 stonith-ng: notice:
crm_cluster_connect: Connecting to cluster infrastructure: corosync
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31932-21-header
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31932-21-header
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31932-21-header
Oct 12 18:41:05 [31932] pm1 attrd: info: crm_get_peer: Created
entry 202bb6fc-5fd5-4506-95cf-13ff576c42d3/0xb22dc0 for node pm1/1 (1 total)
Oct 12 18:41:05 [31932] pm1 attrd: info: crm_get_peer: Node 1
is now known as pm1
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31930-22-header
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31930-22-header
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31930-22-header
Oct 12 18:41:05 [31930] pm1 stonith-ng: info: crm_get_peer: Created
entry b9e3bca1-b31f-49cb-bf0c-88f759827ed8/0x18edde0 for node pm1/1 (1
total)
Oct 12 18:41:05 [31930] pm1 stonith-ng: info: crm_get_peer: Node 1
is now known as pm1
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31932-21-header
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31932-21-header
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31932-21-header
Oct 12 18:41:05 [31932] pm1 attrd: info: crm_get_peer: Node 1
has uuid 1
Oct 12 18:41:05 [31932] pm1 attrd: info:
crm_update_peer_proc: cluster_connect_cpg: Node pm1[1] - corosync-cpg
is now online
Oct 12 18:41:05 [31932] pm1 attrd: notice:
crm_update_peer_state: crm_update_peer_proc: Node pm1[1] - state is
now member (was (null))
Oct 12 18:41:05 [31932] pm1 attrd: info:
init_cs_connection_once: Connection to 'corosync': established
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: activateCibXml:
Triggering CIB write for start op
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31930-22-header
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31930-22-header
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31930-22-header
Oct 12 18:41:05 [31930] pm1 stonith-ng: info: crm_get_peer: Node 1
has uuid 1
Oct 12 18:41:05 [31930] pm1 stonith-ng: info:
crm_update_peer_proc: cluster_connect_cpg: Node pm1[1] - corosync-cpg
is now online
Oct 12 18:41:05 [31930] pm1 stonith-ng: notice:
crm_update_peer_state: crm_update_peer_proc: Node pm1[1] - state is
now member (was (null))
Oct 12 18:41:05 [31930] pm1 stonith-ng: info:
init_cs_connection_once: Connection to 'corosync': established
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31932-21-header
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31932-21-header
Oct 12 18:41:05 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31932-21-header
Oct 12 18:41:05 [31932] pm1 attrd: info: main: Cluster
connection active
Oct 12 18:41:05 [31932] pm1 attrd: info: qb_ipcs_us_publish:
server name: attrd
Oct 12 18:41:05 [31932] pm1 attrd: info: main: Accepting
attribute updates
Oct 12 18:41:05 [31932] pm1 attrd: debug: attrd_cib_connect:
CIB signon attempt 1
Oct 12 18:41:05 [31932] pm1 attrd: debug: crm_ipc_connect:
Could not establish cib_rw connection: Connection refused (111)
Oct 12 18:41:05 [31932] pm1 attrd: debug:
cib_native_signon_raw: Connection unsuccessful (0 (nil))
Oct 12 18:41:05 [31932] pm1 attrd: debug:
cib_native_signon_raw: Connection to CIB failed: Transport endpoint
is not connected
Oct 12 18:41:05 [31932] pm1 attrd: debug: cib_native_signoff:
Signing out of the CIB Service
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31930-22-header
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31930-22-header
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31930-22-header
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: crm_ipc_connect:
Could not establish cib_rw connection: Connection refused (111)
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug:
cib_native_signon_raw: Connection unsuccessful (0 (nil))
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug:
cib_native_signon_raw: Connection to CIB failed: Transport endpoint
is not connected
Oct 12 18:41:05 [31930] pm1 stonith-ng: debug: cib_native_signoff:
Signing out of the CIB Service
Oct 12 18:41:05 [31929] pm1 cib: info: startCib: CIB
Initialization completed successfully
Oct 12 18:41:05 [31929] pm1 cib: notice:
crm_cluster_connect: Connecting to cluster infrastructure: corosync
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: info: crm_get_peer: Created
entry 7bd2fc10-ebe9-4896-b919-63cf6283baab/0xe7ac80 for node pm1/1 (1 total)
Oct 12 18:41:05 [31929] pm1 cib: info: crm_get_peer: Node 1
is now known as pm1
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: info: crm_get_peer: Node 1
has uuid 1
Oct 12 18:41:05 [31929] pm1 cib: info:
crm_update_peer_proc: cluster_connect_cpg: Node pm1[1] - corosync-cpg
is now online
Oct 12 18:41:05 [31929] pm1 cib: notice:
crm_update_peer_state: crm_update_peer_proc: Node pm1[1] - state is
now member (was (null))
Oct 12 18:41:05 [31929] pm1 cib: info:
init_cs_connection_once: Connection to 'corosync': established
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: info: qb_ipcs_us_publish:
server name: cib_ro
Oct 12 18:41:05 [31929] pm1 cib: info: qb_ipcs_us_publish:
server name: cib_rw
Oct 12 18:41:05 [31929] pm1 cib: info: qb_ipcs_us_publish:
server name: cib_shm
Oct 12 18:41:05 [31929] pm1 cib: info: cib_init: Starting cib
mainloop
Oct 12 18:41:05 [31929] pm1 cib: info:
pcmk_cpg_membership: Node 1 joined group cib (counter=0.0)
Oct 12 18:41:05 [31929] pm1 cib: info:
pcmk_cpg_membership: Node 1 still member of group cib (peer=pm1,
counter=0.0)
Oct 12 18:41:05 [31929] pm1 cib: debug: child_waitpid:
wait(31935) = 0: Resource temporarily unavailable (11)
Oct 12 18:41:05 [31929] pm1 cib: info: cib_file_backup:
Archived previous version as /var/lib/pacemaker/cib/cib-15.raw
Oct 12 18:41:05 [31929] pm1 cib: debug:
cib_file_write_with_digest: Writing CIB to disk
Oct 12 18:41:05 [31929] pm1 cib: info:
cib_file_write_with_digest: Wrote version 0.19.0 of the CIB to disk
(digest: 7bbb0ed9ab286c460934a59160e26535)
Oct 12 18:41:05 [31929] pm1 cib: debug:
cib_file_write_with_digest: Wrote digest
7bbb0ed9ab286c460934a59160e26535 to disk
Oct 12 18:41:05 [31929] pm1 cib: info:
cib_file_write_with_digest: Reading cluster configuration file
/var/lib/pacemaker/cib/cib.pv4gLt (digest:
/var/lib/pacemaker/cib/cib.rcOjlG)
Oct 12 18:41:05 [31929] pm1 cib: debug:
cib_file_write_with_digest: Activating /var/lib/pacemaker/cib/cib.pv4gLt
Oct 12 18:41:05 [31929] pm1 cib: info:
pcmk_cpg_membership: Node 2 joined group cib (counter=1.0)
Oct 12 18:41:05 [31929] pm1 cib: info:
pcmk_cpg_membership: Node 1 still member of group cib (peer=pm1,
counter=1.0)
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31929-22-header
Oct 12 18:41:05 [31929] pm1 cib: info: crm_get_peer: Created
entry a482e787-279e-4b46-9310-cb4e2b55c9cf/0xe7e450 for node pm2/2 (2 total)
Oct 12 18:41:05 [31929] pm1 cib: info: crm_get_peer: Node 2
is now known as pm2
Oct 12 18:41:05 [31929] pm1 cib: info: crm_get_peer: Node 2
has uuid 2
Oct 12 18:41:05 [31929] pm1 cib: info:
pcmk_cpg_membership: Node 2 still member of group cib (peer=pm2,
counter=1.1)
Oct 12 18:41:05 [31929] pm1 cib: info:
crm_update_peer_proc: pcmk_cpg_membership: Node pm2[2] - corosync-cpg
is now online
Oct 12 18:41:05 [31929] pm1 cib: notice:
crm_update_peer_state: crm_update_peer_proc: Node pm2[2] - state is
now member (was (null))
Oct 12 18:41:06 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xe7ea00 for uid=189 gid=189 pid=31934
id=d54a3a44-9d69-4e39-ab4a-51ecd0d36dec
Oct 12 18:41:06 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-31934-11)
Oct 12 18:41:06 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31934]
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31934] pm1 crmd: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:06 [31929] pm1 cib: debug:
cib_common_callback_worker: Setting cib_refresh_notify callbacks for
crmd (d54a3a44-9d69-4e39-ab4a-51ecd0d36dec): on
Oct 12 18:41:06 [31929] pm1 cib: debug:
cib_common_callback_worker: Setting cib_diff_notify callbacks for
crmd (d54a3a44-9d69-4e39-ab4a-51ecd0d36dec): on
Oct 12 18:41:06 [31934] pm1 crmd: info: do_cib_control: CIB
connection established
Oct 12 18:41:06 [31934] pm1 crmd: notice:
crm_cluster_connect: Connecting to cluster infrastructure: corosync
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31934-23-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31934-23-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31934-23-header
Oct 12 18:41:06 [31934] pm1 crmd: info: crm_get_peer: Created
entry c80b6ca8-f027-45fa-ab87-ab07daeb829c/0x2133e20 for node pm1/1 (1
total)
Oct 12 18:41:06 [31934] pm1 crmd: info: crm_get_peer: Node 1
is now known as pm1
Oct 12 18:41:06 [31934] pm1 crmd: info:
peer_update_callback: pm1 is now in unknown state
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31934-23-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31934-23-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31934-23-header
Oct 12 18:41:06 [31934] pm1 crmd: info: crm_get_peer: Node 1
has uuid 1
Oct 12 18:41:06 [31934] pm1 crmd: info:
crm_update_peer_proc: cluster_connect_cpg: Node pm1[1] - corosync-cpg
is now online
Oct 12 18:41:06 [31934] pm1 crmd: info:
peer_update_callback: Client pm1/peer now has status [online]
(DC=<null>, changed=4000000)
Oct 12 18:41:06 [31934] pm1 crmd: info:
init_cs_connection_once: Connection to 'corosync': established
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31934-23-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31934-23-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31934-23-header
Oct 12 18:41:06 [31934] pm1 crmd: debug:
cluster_connect_quorum: Configuring Pacemaker to obtain quorum from
Corosync
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: notice:
cluster_connect_quorum: Quorum acquired
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: info: crm_get_peer: Created
entry 1ca36969-6906-4f50-8496-4482ddb87eed/0x2137c00 for node pm2/2 (2
total)
Oct 12 18:41:06 [31934] pm1 crmd: info: crm_get_peer: Node 2
is now known as pm2
Oct 12 18:41:06 [31934] pm1 crmd: info:
peer_update_callback: pm2 is now in unknown state
Oct 12 18:41:06 [31934] pm1 crmd: info: crm_get_peer: Node 2
has uuid 2
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31934-24-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31934-24-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31934-24-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31934-24-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31934-24-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31934-24-header
Oct 12 18:41:06 [31934] pm1 crmd: info: do_ha_control:
Connected to the cluster
Oct 12 18:41:06 [31934] pm1 crmd: debug: do_lrm_control:
Connecting to the LRM
Oct 12 18:41:06 [31934] pm1 crmd: info: lrmd_ipc_connect:
Connecting to lrmd
Oct 12 18:41:06 [31931] pm1 lrmd: debug: crm_client_new:
Connecting 0x1f68140 for uid=189 gid=189 pid=31934
id=767e1a8c-765b-42f5-92f3-5db5fb068a58
Oct 12 18:41:06 [31931] pm1 lrmd: debug:
handle_new_connection: IPC credentials authenticated (31931-31934-8)
Oct 12 18:41:06 [31931] pm1 lrmd: debug:
qb_ipcs_shm_connect: connecting to client [31934]
Oct 12 18:41:06 [31931] pm1 lrmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:06 [31931] pm1 lrmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:06 [31931] pm1 lrmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:06 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:06 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: /cib/configuration/nodes/node[@uname='pm1'][@id!='1'] does
not exist
Oct 12 18:41:06 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:06 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: /cib/configuration/nodes/node[@uname='pm2'][@id!='2'] does
not exist
Oct 12 18:41:06 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
nodes to master (origin=local/crmd/5)
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31929-24-header
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31929-24-header
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31929-24-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:06 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed register operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=0, reply=0, notify=0, exit=4203648
Oct 12 18:41:06 [31934] pm1 crmd: info: do_lrm_control: LRM
connection established
Oct 12 18:41:06 [31934] pm1 crmd: info: do_started: Delaying
start, no membership data (0000000000100000)
Oct 12 18:41:06 [31934] pm1 crmd: debug:
register_fsa_input_adv: Stalling the FSA pending further input:
source=do_started cause=C_FSA_INTERNAL data=(nil) queue=0
Oct 12 18:41:06 [31934] pm1 crmd: debug: s_crmd_fsa: Exiting
the FSA: queue=0, fsa_actions=0x2, stalled=true
Oct 12 18:41:06 [31934] pm1 crmd: info: do_started: Delaying
start, no membership data (0000000000100000)
Oct 12 18:41:06 [31934] pm1 crmd: debug:
register_fsa_input_adv: Stalling the FSA pending further input:
source=do_started cause=C_FSA_INTERNAL data=(nil) queue=0
Oct 12 18:41:06 [31934] pm1 crmd: debug: s_crmd_fsa: Exiting
the FSA: queue=0, fsa_actions=0x2, stalled=true
Oct 12 18:41:06 [31934] pm1 crmd: info:
pcmk_quorum_notification: Membership 172: quorum retained (2)
Oct 12 18:41:06 [31934] pm1 crmd: debug:
pcmk_quorum_notification: Member[0] 1
Oct 12 18:41:06 [31934] pm1 crmd: notice:
crm_update_peer_state: pcmk_quorum_notification: Node pm1[1] - state
is now member (was (null))
Oct 12 18:41:06 [31934] pm1 crmd: info:
peer_update_callback: pm1 is now member (was in unknown state)
Oct 12 18:41:06 [31934] pm1 crmd: debug:
pcmk_quorum_notification: Member[1] 2
Oct 12 18:41:06 [31934] pm1 crmd: notice:
crm_update_peer_state: pcmk_quorum_notification: Node pm2[2] - state
is now member (was (null))
Oct 12 18:41:06 [31934] pm1 crmd: info:
peer_update_callback: pm2 is now member (was in unknown state)
Oct 12 18:41:06 [31934] pm1 crmd: debug: post_cache_update:
Updated cache after membership event 172.
Oct 12 18:41:06 [31934] pm1 crmd: debug: post_cache_update:
post_cache_update added action A_ELECTION_CHECK to the FSA
Oct 12 18:41:06 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:06 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
nodes: OK (rc=0, origin=pm1/crmd/5, version=0.19.0)
Oct 12 18:41:06 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31934-24-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31934-24-header
Oct 12 18:41:06 [31934] pm1 crmd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31934-24-header
Oct 12 18:41:06 [31934] pm1 crmd: info: do_started: Delaying
start, Config not read (0000000000000040)
Oct 12 18:41:06 [31934] pm1 crmd: debug:
register_fsa_input_adv: Stalling the FSA pending further input:
source=do_started cause=C_FSA_INTERNAL data=(nil) queue=0
Oct 12 18:41:06 [31934] pm1 crmd: debug: s_crmd_fsa: Exiting
the FSA: queue=0, fsa_actions=0x200000002, stalled=true
Oct 12 18:41:06 [31934] pm1 crmd: debug:
config_query_callback: Call 6 : Parsing CIB options
Oct 12 18:41:06 [31934] pm1 crmd: debug:
config_query_callback: Shutdown escalation occurs after: 1200000ms
Oct 12 18:41:06 [31934] pm1 crmd: debug:
config_query_callback: Checking for expired actions every 900000ms
Oct 12 18:41:06 [31934] pm1 crmd: debug: do_started: Init
server comms
Oct 12 18:41:06 [31934] pm1 crmd: info: qb_ipcs_us_publish:
server name: crmd
Oct 12 18:41:06 [31934] pm1 crmd: notice: do_started: The
local CRM is operational
Oct 12 18:41:06 [31934] pm1 crmd: debug: do_election_check:
Ignore election check: we not in an election
Oct 12 18:41:06 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PENDING: [ state=S_STARTING cause=C_FSA_INTERNAL origin=do_started ]
Oct 12 18:41:06 [31934] pm1 crmd: info: do_log: FSA: Input
I_PENDING from do_started() received in state S_STARTING
Oct 12 18:41:06 [31934] pm1 crmd: notice:
do_state_transition: State transition S_STARTING -> S_PENDING [
input=I_PENDING cause=C_FSA_INTERNAL origin=do_started ]
Oct 12 18:41:06 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:06 [31932] pm1 attrd: debug: attrd_cib_connect:
CIB signon attempt 2
Oct 12 18:41:06 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf04950 for uid=189 gid=189 pid=31932
id=ed3d3118-6dc6-4da3-b8db-a4d8d5d886a7
Oct 12 18:41:06 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-31932-12)
Oct 12 18:41:06 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31932]
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xe74480 for uid=0 gid=0 pid=31930
id=6f0eace0-28f1-42c5-b600-0b2d173f8e5a
Oct 12 18:41:06 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-31930-13)
Oct 12 18:41:06 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [31930]
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:06 [31932] pm1 attrd: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:06 [31932] pm1 attrd: info: attrd_cib_connect:
Connected to the CIB after 2 attempts
Oct 12 18:41:06 [31929] pm1 cib: debug:
cib_common_callback_worker: Setting cib_refresh_notify callbacks for
attrd (ed3d3118-6dc6-4da3-b8db-a4d8d5d886a7): on
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:06 [31932] pm1 attrd: info: main: CIB
connection active
Oct 12 18:41:06 [31932] pm1 attrd: info:
pcmk_cpg_membership: Node 1 joined group attrd (counter=0.0)
Oct 12 18:41:06 [31932] pm1 attrd: info:
pcmk_cpg_membership: Node 1 still member of group attrd (peer=pm1,
counter=0.0)
Oct 12 18:41:06 [31932] pm1 attrd: info:
pcmk_cpg_membership: Node 2 joined group attrd (counter=1.0)
Oct 12 18:41:06 [31932] pm1 attrd: info:
pcmk_cpg_membership: Node 1 still member of group attrd (peer=pm1,
counter=1.0)
Oct 12 18:41:06 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31932] pm1 attrd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:06 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31932-24-header
Oct 12 18:41:06 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31932-24-header
Oct 12 18:41:06 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31932-24-header
Oct 12 18:41:06 [31932] pm1 attrd: info: crm_get_peer: Created
entry d85a6347-ed6b-47a7-9cbb-77b22766c16c/0xba6020 for node pm2/2 (2 total)
Oct 12 18:41:06 [31932] pm1 attrd: info: crm_get_peer: Node 2
is now known as pm2
Oct 12 18:41:06 [31932] pm1 attrd: info: crm_get_peer: Node 2
has uuid 2
Oct 12 18:41:06 [31932] pm1 attrd: info:
pcmk_cpg_membership: Node 2 still member of group attrd (peer=pm2,
counter=1.1)
Oct 12 18:41:06 [31932] pm1 attrd: info:
crm_update_peer_proc: pcmk_cpg_membership: Node pm2[2] - corosync-cpg
is now online
Oct 12 18:41:06 [31932] pm1 attrd: notice:
crm_update_peer_state: crm_update_peer_proc: Node pm2[2] - state is
now member (was (null))
Oct 12 18:41:06 [31929] pm1 cib: debug:
cib_common_callback_worker: Setting cib_diff_notify callbacks for
crmd (6f0eace0-28f1-42c5-b600-0b2d173f8e5a): on
Oct 12 18:41:06 [31930] pm1 stonith-ng: notice: setup_cib: Watching
for stonith topology changes
Oct 12 18:41:06 [31930] pm1 stonith-ng: info: qb_ipcs_us_publish:
server name: stonith-ng
Oct 12 18:41:06 [31930] pm1 stonith-ng: info: main: Starting
stonith-ng mainloop
Oct 12 18:41:06 [31930] pm1 stonith-ng: info:
pcmk_cpg_membership: Node 1 joined group stonith-ng (counter=0.0)
Oct 12 18:41:06 [31930] pm1 stonith-ng: info:
pcmk_cpg_membership: Node 1 still member of group stonith-ng
(peer=pm1, counter=0.0)
Oct 12 18:41:06 [31930] pm1 stonith-ng: info:
pcmk_cpg_membership: Node 2 joined group stonith-ng (counter=1.0)
Oct 12 18:41:06 [31930] pm1 stonith-ng: info:
pcmk_cpg_membership: Node 1 still member of group stonith-ng
(peer=pm1, counter=1.0)
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31930-24-header
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31930-24-header
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31930-24-header
Oct 12 18:41:06 [31930] pm1 stonith-ng: info: crm_get_peer: Created
entry f4d639dc-9474-41fc-98f6-a36fa7b14498/0x18ef310 for node pm2/2 (2
total)
Oct 12 18:41:06 [31930] pm1 stonith-ng: info: crm_get_peer: Node 2
is now known as pm2
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug:
st_peer_update_callback: Broadcasting our uname because of node 2
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31930-24-header
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31930-24-header
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31930-24-header
Oct 12 18:41:06 [31930] pm1 stonith-ng: info: crm_get_peer: Node 2
has uuid 2
Oct 12 18:41:06 [31930] pm1 stonith-ng: info:
pcmk_cpg_membership: Node 2 still member of group stonith-ng
(peer=pm2, counter=1.1)
Oct 12 18:41:06 [31930] pm1 stonith-ng: info:
crm_update_peer_proc: pcmk_cpg_membership: Node pm2[2] - corosync-cpg
is now online
Oct 12 18:41:06 [31930] pm1 stonith-ng: notice:
crm_update_peer_state: crm_update_peer_proc: Node pm2[2] - state is
now member (was (null))
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug:
st_peer_update_callback: Broadcasting our uname because of node 2
Oct 12 18:41:06 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:06 [31930] pm1 stonith-ng: info: init_cib_cache_cb:
Updating device list from the cib: init
Oct 12 18:41:06 [31930] pm1 stonith-ng: info: cib_devices_update:
Updating devices to version 0.19.0
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:06 [31930] pm1 stonith-ng: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:06 [31930] pm1 stonith-ng: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:06 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:07 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
nodes: OK (rc=0, origin=pm2/crmd/5, version=0.19.0)
Oct 12 18:41:07 [31934] pm1 crmd: debug: do_cl_join_query:
Querying for a DC
Oct 12 18:41:07 [31934] pm1 crmd: debug: crm_timer_start:
Started Election Trigger (I_DC_TIMEOUT:20000ms), src=21
Oct 12 18:41:07 [31934] pm1 crmd: info:
pcmk_cpg_membership: Node 1 joined group crmd (counter=0.0)
Oct 12 18:41:07 [31934] pm1 crmd: info:
pcmk_cpg_membership: Node 1 still member of group crmd (peer=pm1,
counter=0.0)
Oct 12 18:41:07 [31934] pm1 crmd: info:
pcmk_cpg_membership: Node 2 joined group crmd (counter=1.0)
Oct 12 18:41:07 [31934] pm1 crmd: info:
pcmk_cpg_membership: Node 1 still member of group crmd (peer=pm1,
counter=1.0)
Oct 12 18:41:07 [31934] pm1 crmd: info:
pcmk_cpg_membership: Node 2 still member of group crmd (peer=pm2,
counter=1.1)
Oct 12 18:41:07 [31934] pm1 crmd: info:
crm_update_peer_proc: pcmk_cpg_membership: Node pm2[2] - corosync-cpg
is now online
Oct 12 18:41:07 [31934] pm1 crmd: info:
peer_update_callback: Client pm2/peer now has status [online]
(DC=<null>, changed=4000000)
Oct 12 18:41:07 [31934] pm1 crmd: debug: te_connect_stonith:
Attempting connection to fencing daemon...
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: crm_client_new:
Connecting 0x1929620 for uid=189 gid=189 pid=31934
id=b890d981-0ecb-4e6d-8020-2ffda2a7db0a
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug:
handle_new_connection: IPC credentials authenticated (31930-31934-10)
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug:
qb_ipcs_shm_connect: connecting to client [31934]
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:08 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:08 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:08 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: stonith_command:
Processing register 11 from crmd.31934 ( 0)
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: stonith_command:
Processed register from crmd.31934: OK (0)
Oct 12 18:41:08 [31934] pm1 crmd: debug: stonith_api_signon:
Connection to STONITH successful
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: stonith_command:
Processing st_notify 12 from crmd.31934 ( 0)
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: handle_request:
Setting st_notify_disconnect callbacks for crmd.31934
(b890d981-0ecb-4e6d-8020-2ffda2a7db0a): ON
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: stonith_command:
Processed st_notify from crmd.31934: OK (0)
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: stonith_command:
Processing st_notify 13 from crmd.31934 ( 0)
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: handle_request:
Setting st_notify_fence callbacks for crmd.31934
(b890d981-0ecb-4e6d-8020-2ffda2a7db0a): ON
Oct 12 18:41:08 [31930] pm1 stonith-ng: debug: stonith_command:
Processed st_notify from crmd.31934: OK (0)
Oct 12 18:41:27 [31934] pm1 crmd: info: crm_timer_popped:
Election Trigger (I_DC_TIMEOUT) just popped (20000ms)
Oct 12 18:41:27 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_DC_TIMEOUT: [ state=S_PENDING cause=C_TIMER_POPPED
origin=crm_timer_popped ]
Oct 12 18:41:27 [31934] pm1 crmd: warning: do_log: FSA: Input
I_DC_TIMEOUT from crm_timer_popped() received in state S_PENDING
Oct 12 18:41:27 [31934] pm1 crmd: info:
do_state_transition: State transition S_PENDING -> S_ELECTION [
input=I_DC_TIMEOUT cause=C_TIMER_POPPED origin=crm_timer_popped ]
Oct 12 18:41:27 [31934] pm1 crmd: debug: crm_uptime: Current
CPU usage is: 0s, 13027us
Oct 12 18:41:27 [31934] pm1 crmd: debug: election_vote: Started
election 1
Oct 12 18:41:27 [31934] pm1 crmd: debug:
election_count_vote: Created voted hash
Oct 12 18:41:27 [31934] pm1 crmd: debug:
election_count_vote: Election 1 (current: 1, owner: 1): Processed
vote from pm1 (Recorded)
Oct 12 18:41:27 [31934] pm1 crmd: debug: election_check: Still
waiting on 1 non-votes (2 total)
Oct 12 18:41:27 [31934] pm1 crmd: debug:
election_count_vote: Election 1 (current: 1, owner: 1): Processed
no-vote from pm2 (Recorded)
Oct 12 18:41:27 [31934] pm1 crmd: info: election_complete:
Election election-0 complete
Oct 12 18:41:27 [31934] pm1 crmd: info:
election_timeout_popped: Election failed: Declaring ourselves the winner
Oct 12 18:41:27 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_ELECTION_DC: [ state=S_ELECTION cause=C_TIMER_POPPED
origin=election_timeout_popped ]
Oct 12 18:41:27 [31934] pm1 crmd: info: do_log: FSA: Input
I_ELECTION_DC from election_timeout_popped() received in state S_ELECTION
Oct 12 18:41:27 [31934] pm1 crmd: notice:
do_state_transition: State transition S_ELECTION -> S_INTEGRATION [
input=I_ELECTION_DC cause=C_TIMER_POPPED origin=election_timeout_popped ]
Oct 12 18:41:27 [31934] pm1 crmd: info: do_te_control:
Registering TE UUID: b020e8a2-0497-404a-b66a-e629cc7d3213
Oct 12 18:41:27 [31929] pm1 cib: debug:
cib_common_callback_worker: Setting cib_diff_notify callbacks for
crmd (d54a3a44-9d69-4e39-ab4a-51ecd0d36dec): on
Oct 12 18:41:27 [31934] pm1 crmd: info:
set_graph_functions: Setting custom graph functions
Oct 12 18:41:27 [31934] pm1 crmd: debug: do_te_control:
Transitioner is now active
Oct 12 18:41:27 [31934] pm1 crmd: debug: unpack_graph: Unpacked
transition -1: 0 actions in 0 synapses
Oct 12 18:41:27 [31933] pm1 pengine: debug: crm_client_new:
Connecting 0x18bad10 for uid=189 gid=189 pid=31934
id=a9f76238-7236-441c-bfd3-801fe84456a2
Oct 12 18:41:27 [31933] pm1 pengine: debug:
handle_new_connection: IPC credentials authenticated (31933-31934-8)
Oct 12 18:41:27 [31933] pm1 pengine: debug:
qb_ipcs_shm_connect: connecting to client [31934]
Oct 12 18:41:27 [31933] pm1 pengine: debug: qb_rb_open_2: shm
size:5242893; real_size:5246976; rb->word_size:1311744
Oct 12 18:41:27 [31933] pm1 pengine: debug: qb_rb_open_2: shm
size:5242893; real_size:5246976; rb->word_size:1311744
Oct 12 18:41:27 [31933] pm1 pengine: debug: qb_rb_open_2: shm
size:5242893; real_size:5246976; rb->word_size:1311744
Oct 12 18:41:27 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:5242893; real_size:5246976; rb->word_size:1311744
Oct 12 18:41:27 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:5242893; real_size:5246976; rb->word_size:1311744
Oct 12 18:41:27 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:5242893; real_size:5246976; rb->word_size:1311744
Oct 12 18:41:27 [31934] pm1 crmd: debug: crm_timer_start:
Started Integration Timer (I_INTEGRATED:180000ms), src=25
Oct 12 18:41:27 [31934] pm1 crmd: info: do_dc_takeover: Taking
over DC status for this partition
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31929] pm1 cib: info:
cib_process_readwrite: We are now in R/W mode
Oct 12 18:41:27 [31929] pm1 cib: info:
cib_process_request: Completed cib_master operation for section
'all': OK (rc=0, origin=local/crmd/8, version=0.19.0)
Oct 12 18:41:27 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section cib
to master (origin=local/crmd/9)
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/configuration/crm_config//cluster_property_set//nvpair[@name='have-watchdog']
(/cib/configuration/crm_config/cluster_property_set/nvpair[1])
Oct 12 18:41:27 [31934] pm1 crmd: debug:
find_nvpair_attr_delegate: Match <nvpair
id="cib-bootstrap-options-have-watchdog" name="have-watchdog"
value="false"/>
Oct 12 18:41:27 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
crm_config to master (origin=local/crmd/11)
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/configuration/crm_config//cluster_property_set//nvpair[@name='dc-version']
(/cib/configuration/crm_config/cluster_property_set/nvpair[2])
Oct 12 18:41:27 [31934] pm1 crmd: debug:
find_nvpair_attr_delegate: Match <nvpair
id="cib-bootstrap-options-dc-version" name="dc-version"
value="1.1.13-a14efad"/>
Oct 12 18:41:27 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
crm_config to master (origin=local/crmd/13)
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/configuration/crm_config//cluster_property_set//nvpair[@name='cluster-infrastructure']
(/cib/configuration/crm_config/cluster_property_set/nvpair[3])
Oct 12 18:41:27 [31934] pm1 crmd: debug:
find_nvpair_attr_delegate: Match <nvpair
id="cib-bootstrap-options-cluster-infrastructure"
name="cluster-infrastructure" value="corosync"/>
Oct 12 18:41:27 [31934] pm1 crmd: debug: initialize_join:
join-1: Initializing join data (flag=true)
Oct 12 18:41:27 [31934] pm1 crmd: info: join_make_offer:
Making join offers based on membership 172
Oct 12 18:41:27 [31934] pm1 crmd: info: join_make_offer:
join-1: Sending offer to pm1
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_join: join_make_offer: Node pm1[1] - join-1 phase 0 -> 1
Oct 12 18:41:27 [31934] pm1 crmd: info: join_make_offer:
join-1: Sending offer to pm2
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_join: join_make_offer: Node pm2[2] - join-1 phase 0 -> 1
Oct 12 18:41:27 [31934] pm1 crmd: info:
do_dc_join_offer_all: join-1: Waiting on 2 outstanding join acks
Oct 12 18:41:27 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_ELECTION_DC: [ state=S_INTEGRATION cause=C_FSA_INTERNAL
origin=do_election_check ]
Oct 12 18:41:27 [31934] pm1 crmd: warning: do_log: FSA: Input
I_ELECTION_DC from do_election_check() received in state S_INTEGRATION
Oct 12 18:41:27 [31934] pm1 crmd: debug: election_vote: Started
election 2
Oct 12 18:41:27 [31934] pm1 crmd: debug: initialize_join:
join-2: Initializing join data (flag=true)
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_join: initialize_join: Node pm1[1] - join-2 phase 1 -> 0
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_join: initialize_join: Node pm2[2] - join-2 phase 1 -> 0
Oct 12 18:41:27 [31934] pm1 crmd: info: join_make_offer:
join-2: Sending offer to pm1
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_join: join_make_offer: Node pm1[1] - join-2 phase 0 -> 1
Oct 12 18:41:27 [31934] pm1 crmd: info: join_make_offer:
join-2: Sending offer to pm2
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_join: join_make_offer: Node pm2[2] - join-2 phase 0 -> 1
Oct 12 18:41:27 [31934] pm1 crmd: info:
do_dc_join_offer_all: join-2: Waiting on 2 outstanding join acks
Oct 12 18:41:27 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
crm_config to master (origin=local/crmd/15)
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31934] pm1 crmd: debug:
config_query_callback: Call 16 : Parsing CIB options
Oct 12 18:41:27 [31934] pm1 crmd: debug:
config_query_callback: Shutdown escalation occurs after: 1200000ms
Oct 12 18:41:27 [31934] pm1 crmd: debug:
config_query_callback: Checking for expired actions every 900000ms
Oct 12 18:41:27 [31934] pm1 crmd: debug: handle_request:
Raising I_JOIN_OFFER: join-1
Oct 12 18:41:27 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_JOIN_OFFER: [ state=S_INTEGRATION cause=C_HA_MESSAGE
origin=route_message ]
Oct 12 18:41:27 [31934] pm1 crmd: info: update_dc: Set DC
to pm1 (3.0.10)
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_expected: update_dc: Node pm1[1] - expected state is
now member (was (null))
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_cl_join_offer_respond: do_cl_join_offer_respond added action
A_DC_TIMER_STOP to the FSA
Oct 12 18:41:27 [31934] pm1 crmd: debug:
election_count_vote: Created voted hash
Oct 12 18:41:27 [31934] pm1 crmd: debug:
election_count_vote: Election 2 (current: 2, owner: 1): Processed
vote from pm1 (Recorded)
Oct 12 18:41:27 [31934] pm1 crmd: debug: do_election_check:
Ignore election check: we not in an election
Oct 12 18:41:27 [31934] pm1 crmd: debug: handle_request:
Raising I_JOIN_OFFER: join-2
Oct 12 18:41:27 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_JOIN_OFFER: [ state=S_INTEGRATION cause=C_HA_MESSAGE
origin=route_message ]
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_cl_join_offer_respond: do_cl_join_offer_respond added action
A_DC_TIMER_STOP to the FSA
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31934] pm1 crmd: debug:
election_count_vote: Election 2 (current: 2, owner: 1): Processed
no-vote from pm2 (Recorded)
Oct 12 18:41:27 [31934] pm1 crmd: debug: do_election_check:
Ignore election check: we not in an election
Oct 12 18:41:27 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section cib:
OK (rc=0, origin=pm1/crmd/9, version=0.19.0)
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
crm_config: OK (rc=0, origin=pm1/crmd/11, version=0.19.0)
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
crm_config: OK (rc=0, origin=pm1/crmd/13, version=0.19.0)
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
crm_config: OK (rc=0, origin=pm1/crmd/15, version=0.19.0)
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31934] pm1 crmd: debug:
join_query_callback: Respond to join offer join-2 from pm1
Oct 12 18:41:27 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_JOIN_REQUEST: [ state=S_INTEGRATION cause=C_HA_MESSAGE
origin=route_message ]
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_filter_offer: Processing req from pm2
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_filter_offer: join-2: Welcoming node pm2 (ref
join_request-crmd-1444664487-4)
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_join: do_dc_join_filter_offer: Node pm2[2] - join-2
phase 1 -> 2
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_expected: do_dc_join_filter_offer: Node pm2[2] -
expected state is now member (was (null))
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_filter_offer: 1 nodes have been integrated into join-2
Oct 12 18:41:27 [31934] pm1 crmd: debug: check_join_state:
Invoked by do_dc_join_filter_offer in state: S_INTEGRATION
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_filter_offer: join-2: Still waiting on 1 outstanding offers
Oct 12 18:41:27 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_JOIN_REQUEST: [ state=S_INTEGRATION cause=C_HA_MESSAGE
origin=route_message ]
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_filter_offer: Processing req from pm1
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_filter_offer: pm1 has a better generation number than the
current max pm2
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_filter_offer: Max generation <generation_tuple
crm_feature_set="3.0.10" validate-with="pacemaker-2.3" epoch="19"
num_updates="0" admin_epoch="0" cib-last-written="Mon Oct 12 18:40:24
2015" update-origin="pm1" update-client="crm_attribute"
update-user="root" have-quorum="1"/>
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_filter_offer: Their generation <generation_tuple
crm_feature_set="3.0.10" validate-with="pacemaker-2.3" epoch="19"
num_updates="0" admin_epoch="0" cib-last-written="Mon Oct 12 18:31:10
2015" update-origin="pm1" update-client="crm_attribute"
update-user="root" have-quorum="1"/>
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_filter_offer: join-2: Welcoming node pm1 (ref
join_request-crmd-1444664487-8)
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_join: do_dc_join_filter_offer: Node pm1[1] - join-2
phase 1 -> 2
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_filter_offer: 2 nodes have been integrated into join-2
Oct 12 18:41:27 [31934] pm1 crmd: debug: check_join_state:
Invoked by do_dc_join_filter_offer in state: S_INTEGRATION
Oct 12 18:41:27 [31934] pm1 crmd: debug: check_join_state:
join-2: Integration of 2 peers complete: do_dc_join_filter_offer
Oct 12 18:41:27 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_INTEGRATED: [ state=S_INTEGRATION cause=C_FSA_INTERNAL
origin=check_join_state ]
Oct 12 18:41:27 [31934] pm1 crmd: info:
do_state_transition: State transition S_INTEGRATION ->
S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL
origin=check_join_state ]
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes responded to the join offer.
Oct 12 18:41:27 [31934] pm1 crmd: debug: crm_timer_start:
Started Finalization Timer (I_ELECTION:1800000ms), src=31
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_finalize: Finializing join-2 for 2 clients
Oct 12 18:41:27 [31934] pm1 crmd: info:
crmd_join_phase_log: join-2: pm1=integrated
Oct 12 18:41:27 [31934] pm1 crmd: info:
crmd_join_phase_log: join-2: pm2=integrated
Oct 12 18:41:27 [31934] pm1 crmd: info:
do_dc_join_finalize: join-2: Syncing our CIB to the rest of the cluster
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_dc_join_finalize: Requested version <generation_tuple
crm_feature_set="3.0.10" validate-with="pacemaker-2.3" epoch="19"
num_updates="0" admin_epoch="0" cib-last-written="Mon Oct 12 18:31:10
2015" update-origin="pm1" update-client="crm_attribute"
update-user="root" have-quorum="1"/>
Oct 12 18:41:27 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:27 [31929] pm1 cib: debug: sync_our_cib: Syncing
CIB to all peers
Oct 12 18:41:27 [31934] pm1 crmd: debug: check_join_state:
Invoked by finalize_sync_callback in state: S_FINALIZE_JOIN
Oct 12 18:41:27 [31934] pm1 crmd: debug: check_join_state:
join-2: Still waiting on 2 integrated nodes
Oct 12 18:41:27 [31934] pm1 crmd: debug:
crmd_join_phase_log: join-2: pm1=integrated
Oct 12 18:41:27 [31934] pm1 crmd: debug:
crmd_join_phase_log: join-2: pm2=integrated
Oct 12 18:41:27 [31934] pm1 crmd: debug:
finalize_sync_callback: Notifying 2 clients of join-2 results
Oct 12 18:41:27 [31934] pm1 crmd: debug: finalize_join_for:
join-2: ACK'ing join request from pm1
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_join: finalize_join_for: Node pm1[1] - join-2 phase 2
-> 3
Oct 12 18:41:27 [31934] pm1 crmd: debug: finalize_join_for:
join-2: ACK'ing join request from pm2
Oct 12 18:41:27 [31934] pm1 crmd: info:
crm_update_peer_join: finalize_join_for: Node pm2[2] - join-2 phase 2
-> 3
Oct 12 18:41:27 [31934] pm1 crmd: debug: handle_request:
Raising I_JOIN_RESULT: join-2
Oct 12 18:41:27 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_JOIN_RESULT: [ state=S_FINALIZE_JOIN cause=C_HA_MESSAGE
origin=route_message ]
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_cl_join_finalize_respond: Confirming join join-2: join_ack_nack
Oct 12 18:41:27 [31934] pm1 crmd: debug:
do_cl_join_finalize_respond: join-2: Join complete. Sending local
LRM status to pm1
Oct 12 18:41:27 [31934] pm1 crmd: info: erase_status_tag:
Deleting xpath: //node_state[@uname='pm1']/transient_attributes
Oct 12 18:41:27 [31934] pm1 crmd: info:
update_attrd_helper: Connecting to attribute manager ... 5 retries
remaining
Oct 12 18:41:27 [31932] pm1 attrd: debug: crm_client_new:
Connecting 0xba7200 for uid=189 gid=189 pid=31934
id=5f9654e7-ab67-421f-83ea-5ad839cf1762
Oct 12 18:41:27 [31932] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (31932-31934-10)
Oct 12 18:41:27 [31932] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [31934]
Oct 12 18:41:27 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:27 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:27 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:27 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:27 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:27 [31934] pm1 crmd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:27 [31932] pm1 attrd: info:
attrd_client_update: Starting an election to determine the writer
Oct 12 18:41:27 [31932] pm1 attrd: debug: crm_uptime: Current
CPU usage is: 0s, 11672us
Oct 12 18:41:27 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:27 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:27 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:27 [31932] pm1 attrd: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:27 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31932-24-header
Oct 12 18:41:27 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31932-24-header
Oct 12 18:41:27 [31932] pm1 attrd: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31932-24-header
Oct 12 18:41:27 [31932] pm1 attrd: debug: election_vote: Started
election 1
Oct 12 18:41:27 [31932] pm1 attrd: debug:
attrd_client_update: Broadcasting terminate[pm1] = (null)
Oct 12 18:41:28 [31932] pm1 attrd: debug:
election_count_vote: Created voted hash
Oct 12 18:41:28 [31932] pm1 attrd: debug: crm_compare_age:
Win: 0.11672 vs 0.6936 (usec)
Oct 12 18:41:28 [31932] pm1 attrd: info:
election_count_vote: Election 1 (owner: 2) pass: vote from pm2 (Uptime)
Oct 12 18:41:28 [31932] pm1 attrd: debug: election_vote: Started
election 2
Oct 12 18:41:27 [31934] pm1 crmd: debug:
attrd_update_delegate: Sent update: terminate=(null) for pm1
Oct 12 18:41:28 [31934] pm1 crmd: debug:
attrd_update_delegate: Sent update: shutdown=0 for pm1
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_dc_join_ack:
Ignoring op=join_ack_nack message from pm1
Oct 12 18:41:28 [31932] pm1 attrd: debug:
election_count_vote: Created voted hash
Oct 12 18:41:28 [31932] pm1 attrd: debug: election_check: Still
waiting on 2 non-votes (2 total)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_peer_update:
Setting shutdown[pm2]: (null) -> 0 from pm2
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_JOIN_RESULT: [ state=S_FINALIZE_JOIN cause=C_HA_MESSAGE
origin=route_message ]
Oct 12 18:41:28 [31934] pm1 crmd: info:
crm_update_peer_join: do_dc_join_ack: Node pm2[2] - join-2 phase 3 -> 4
Oct 12 18:41:28 [31934] pm1 crmd: info: do_dc_join_ack:
join-2: Updating node state to member for pm2
Oct 12 18:41:28 [31934] pm1 crmd: info: erase_status_tag:
Deleting xpath: //node_state[@uname='pm2']/lrm
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_dc_join_ack:
join-2: Registered callback for LRM update 24
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_JOIN_RESULT: [ state=S_FINALIZE_JOIN cause=C_HA_MESSAGE
origin=route_message ]
Oct 12 18:41:28 [31934] pm1 crmd: info:
crm_update_peer_join: do_dc_join_ack: Node pm1[1] - join-2 phase 3 -> 4
Oct 12 18:41:28 [31934] pm1 crmd: info: do_dc_join_ack:
join-2: Updating node state to member for pm1
Oct 12 18:41:28 [31934] pm1 crmd: info: erase_status_tag:
Deleting xpath: //node_state[@uname='pm1']/lrm
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_dc_join_ack:
join-2: Registered callback for LRM update 26
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_replace: Digest matched on replace from pm1:
83e830719e5033f71afcb6a5b0aebc83
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_replace: Replaced 0.19.0 with 0.19.0 from pm1
Oct 12 18:41:28 [31932] pm1 attrd: debug: election_check: Still
waiting on 2 non-votes (2 total)
Oct 12 18:41:28 [31932] pm1 attrd: debug:
election_count_vote: Election 2 (current: 2, owner: 1): Processed
vote from pm1 (Recorded)
Oct 12 18:41:28 [31932] pm1 attrd: debug: election_check: Still
waiting on 1 non-votes (2 total)
Oct 12 18:41:28 [31932] pm1 attrd: debug:
election_count_vote: Election 2 (current: 2, owner: 1): Processed
no-vote from pm2 (Recorded)
Oct 12 18:41:28 [31932] pm1 attrd: info: election_complete:
Election election-attrd complete
Oct 12 18:41:28 [31932] pm1 attrd: debug: attrd_peer_sync:
Syncing shutdown[pm2] = 0 to everyone
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31932] pm1 attrd: debug: attrd_peer_sync:
Syncing terminate[pm1] = (null) to everyone
Oct 12 18:41:28 [31932] pm1 attrd: debug: attrd_peer_sync:
Syncing terminate[pm2] = (null) to everyone
Oct 12 18:41:28 [31932] pm1 attrd: debug: attrd_peer_sync:
Syncing values to everyone
Oct 12 18:41:28 [31929] pm1 cib: debug: activateCibXml:
Triggering CIB write for cib_replace op
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_replace operation for section
'all': OK (rc=0, origin=pm1/crmd/19, version=0.19.0)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_process_xpath:
//node_state[@uname='pm2']/transient_attributes was already removed
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_delete operation for section
//node_state[@uname='pm2']/transient_attributes: OK (rc=0,
origin=pm2/crmd/10, version=0.19.0)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
nodes to master (origin=local/crmd/20)
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm2[shutdown]=0 (2 2 2 pm2)
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Sent update 2 with 1 changes for shutdown, id=<n/a>, set=(null)
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[terminate]=(null) (1 1 1 pm1)
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm2[terminate]=(null) (2 2 2 pm2)
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Sent update 3 with 2 changes for terminate, id=<n/a>, set=(null)
Oct 12 18:41:28 [31932] pm1 attrd: debug:
attrd_client_update: Broadcasting shutdown[pm1] = 0 (writer)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_peer_update:
Setting shutdown[pm1]: (null) -> 0 from pm1
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Write out of 'shutdown' delayed: update 2 in progress
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
nodes to master (origin=local/crmd/21)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_delete operation for section
//node_state[@uname='pm1']/transient_attributes to master
(origin=local/crmd/22)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_delete operation for section
//node_state[@uname='pm2']/lrm to master (origin=local/crmd/23)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/24)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
nodes: OK (rc=0, origin=pm1/crmd/20, version=0.19.0)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
nodes: OK (rc=0, origin=pm1/crmd/21, version=0.19.0)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_process_xpath:
//node_state[@uname='pm1']/transient_attributes was already removed
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_delete operation for section
//node_state[@uname='pm1']/transient_attributes: OK (rc=0,
origin=pm1/crmd/22, version=0.19.0)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_process_xpath:
//node_state[@uname='pm2']/lrm was already removed
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_delete operation for section
//node_state[@uname='pm2']/lrm: OK (rc=0, origin=pm1/crmd/23,
version=0.19.0)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.0 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.1 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=1
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status: <node_state id="2" uname="pm2" in_ccm="true" crmd="online"
crm-debug-origin="do_lrm_query_internal" join="member" expected="member"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ <lrm id="2">
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ <lrm_resources/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ </lrm>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ </node_state>
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/24, version=0.19.1)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_delete operation for section
//node_state[@uname='pm1']/lrm to master (origin=local/crmd/25)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
erase_xpath_callback: Deletion of
"//node_state[@uname='pm1']/transient_attributes": OK (rc=0)
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.1 to 0.19.0
Oct 12 18:41:28 [31934] pm1 crmd: debug:
erase_xpath_callback: Deletion of "//node_state[@uname='pm2']/lrm":
OK (rc=0)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
join_update_complete_callback: Join update 24 complete
Oct 12 18:41:28 [31934] pm1 crmd: debug: check_join_state:
Invoked by join_update_complete_callback in state: S_FINALIZE_JOIN
Oct 12 18:41:28 [31934] pm1 crmd: debug: check_join_state:
join-2 complete: join_update_complete_callback
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_FINALIZED: [ state=S_FINALIZE_JOIN cause=C_FSA_INTERNAL
origin=check_join_state ]
Oct 12 18:41:28 [31934] pm1 crmd: info:
do_state_transition: State transition S_FINALIZE_JOIN ->
S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL
origin=check_join_state ]
Oct 12 18:41:28 [31934] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_dc_join_final:
Ensuring DC, quorum and node attributes are up-to-date
Oct 12 18:41:28 [31934] pm1 crmd: debug:
attrd_update_delegate: Sent update: (null)=(null) for localhost
Oct 12 18:41:28 [31934] pm1 crmd: debug: crm_update_quorum:
Updating quorum status to true (call=31)
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_te_invoke:
Cancelling the transition: inactive
Oct 12 18:41:28 [31934] pm1 crmd: info:
abort_transition_graph: Transition aborted: Peer Cancelled
(source=do_te_invoke:151, 1)
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_POLICY_ENGINE cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_pe_invoke: Query
32: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:41:28 [31932] pm1 attrd: info:
attrd_client_refresh: Updating all attributes
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Write out of 'shutdown' delayed: update 2 in progress
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Write out of 'terminate' delayed: update 3 in progress
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/26)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/2)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/3)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_process_xpath:
//node_state[@uname='pm1']/lrm was already removed
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_delete operation for section
//node_state[@uname='pm1']/lrm: OK (rc=0, origin=pm1/crmd/25,
version=0.19.1)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.1 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.2 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status: <node_state id="1" uname="pm1" in_ccm="true" crmd="online"
crm-debug-origin="do_lrm_query_internal" join="member" expected="member"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ <lrm id="1">
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ <lrm_resources/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ </lrm>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ </node_state>
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/26, version=0.19.2)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.2 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.3 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=3
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='2']: <transient_attributes id="2"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ <instance_attributes id="status-2">
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ <nvpair id="status-2-shutdown"
name="shutdown" value="0"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ </instance_attributes>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ </transient_attributes>
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/2, version=0.19.3)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: /cib/configuration/nodes/node[@uname='pm1'][@id!='1'] does
not exist
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: /cib/configuration/nodes/node[@uname='pm2'][@id!='2'] does
not exist
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
nodes to master (origin=local/crmd/29)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
erase_xpath_callback: Deletion of "//node_state[@uname='pm1']/lrm":
OK (rc=0)
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.1 -> 0.19.2 (S_POLICY_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
join_update_complete_callback: Join update 26 complete
Oct 12 18:41:28 [31934] pm1 crmd: debug: check_join_state:
Invoked by join_update_complete_callback in state: S_POLICY_ENGINE
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.2 -> 0.19.3 (S_POLICY_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info:
abort_transition_graph: Transition aborted by transient_attributes.2
'create': Transient attribute change (cib=0.19.3,
source=te_update_diff:391, path=/cib/status/node_state[@id='2'], 1)
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_POLICY_ENGINE cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_pe_invoke: Query
33: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.2 to 0.19.1
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.3 to 0.19.2
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 2 for shutdown: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 2 for shutdown[pm1]=(null): OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 2 for shutdown[pm2]=0: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[shutdown]=0 (1 1 1 pm1)
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm2[shutdown]=0 (2 2 2 pm2)
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Sent update 4 with 2 changes for shutdown, id=<n/a>, set=(null)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/30)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section cib
to master (origin=local/crmd/31)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_process_modify:
Destroying
/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2]
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_process_modify:
Destroying
/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.3 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.4 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=4
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']: <transient_attributes id="1"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ <instance_attributes id="status-1"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op:
++ </transient_attributes>
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.4 to 0.19.3
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/3, version=0.19.4)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
nodes: OK (rc=0, origin=pm1/crmd/29, version=0.19.4)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.4 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.5 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=5
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='2']: @crm-debug-origin=do_state_transition
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']: @crm-debug-origin=do_state_transition
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/30, version=0.19.5)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.5 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.6 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=6, @dc-uuid=1
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section cib:
OK (rc=0, origin=pm1/crmd/31, version=0.19.6)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/4)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 3 for terminate: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 3 for terminate[pm1]=(null): OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 3 for terminate[pm2]=(null): OK (0)
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.3 -> 0.19.4 (S_POLICY_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info:
abort_transition_graph: Transition aborted by transient_attributes.1
'create': Transient attribute change (cib=0.19.4,
source=te_update_diff:391, path=/cib/status/node_state[@id='1'], 1)
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.4 -> 0.19.5 (S_POLICY_ENGINE)
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.5 to 0.19.4
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.5 -> 0.19.6 (S_POLICY_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=33,
ref=pe_calc-dc-1444664488-12, seq=172, quorate=1
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_POLICY_ENGINE cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_pe_invoke: Query
34: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:28 [31933] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:28 [31933] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:28 [31933] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:28 [31933] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:41:28 [31933] pm1 pengine: info: short_print:
Stopped: [ pm1 pm2 ]
Oct 12 18:41:28 [31933] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:41:28 [31933] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:28 [31933] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:0
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:1
Oct 12 18:41:28 [31933] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:41:28 [31933] pm1 pengine: debug: master_color: pgsql:0
master score: -1
Oct 12 18:41:28 [31933] pm1 pengine: debug: master_color: pgsql:1
master score: -1
Oct 12 18:41:28 [31933] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:41:28 [31933] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:41:28 [31933] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:41:28 [31933] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing pgsql:0 on pm1 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-master on pm1 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-rep on pm1 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing pgsql:1 on pm2 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-master on pm2 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-rep on pm2 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm1
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm2
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm1
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm2
Oct 12 18:41:28 [31933] pm1 pengine: notice: LogActions: Start
pgsql:0 (pm1)
Oct 12 18:41:28 [31933] pm1 pengine: notice: LogActions: Start
pgsql:1 (pm2)
Oct 12 18:41:28 [31933] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: notice: process_pe_message:
Calculated Transition 0: /var/lib/pacemaker/pengine/pe-input-108.bz2
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.6 to 0.19.5
Oct 12 18:41:28 [31934] pm1 crmd: info: handle_response:
pe_calc calculation pe_calc-dc-1444664488-12 is obsolete
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.6 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.7 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=7
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']:
<nvpair id="status-1-shutdown" name="shutdown" value="0"/>
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/4, version=0.19.7)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.7 to 0.19.6
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.6 -> 0.19.7 (S_POLICY_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info:
abort_transition_graph: Transition aborted by status-1-shutdown,
shutdown=0: Transient attribute change (create cib=0.19.7,
source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'],
1)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=34,
ref=pe_calc-dc-1444664488-13, seq=172, quorate=1
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_POLICY_ENGINE cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_pe_invoke: Query
35: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 4 for shutdown: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 4 for shutdown[pm1]=0: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 4 for shutdown[pm2]=0: OK (0)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31934] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=35,
ref=pe_calc-dc-1444664488-14, seq=172, quorate=1
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:28 [31933] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:28 [31933] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:28 [31933] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:28 [31933] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:41:28 [31933] pm1 pengine: info: short_print:
Stopped: [ pm1 pm2 ]
Oct 12 18:41:28 [31933] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:41:28 [31933] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:28 [31933] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:0
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:1
Oct 12 18:41:28 [31933] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:41:28 [31933] pm1 pengine: debug: master_color: pgsql:0
master score: -1
Oct 12 18:41:28 [31933] pm1 pengine: debug: master_color: pgsql:1
master score: -1
Oct 12 18:41:28 [31933] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:41:28 [31933] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:41:28 [31933] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:41:28 [31933] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing pgsql:0 on pm1 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-master on pm1 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-rep on pm1 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing pgsql:1 on pm2 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-master on pm2 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-rep on pm2 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm1
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm2
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm1
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm2
Oct 12 18:41:28 [31933] pm1 pengine: notice: LogActions: Start
pgsql:0 (pm1)
Oct 12 18:41:28 [31933] pm1 pengine: notice: LogActions: Start
pgsql:1 (pm2)
Oct 12 18:41:28 [31933] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: notice: process_pe_message:
Calculated Transition 1: /var/lib/pacemaker/pengine/pe-input-109.bz2
Oct 12 18:41:28 [31934] pm1 crmd: info: handle_response:
pe_calc calculation pe_calc-dc-1444664488-13 is obsolete
Oct 12 18:41:28 [31929] pm1 cib: info: cib_file_backup:
Archived previous version as /var/lib/pacemaker/cib/cib-16.raw
Oct 12 18:41:28 [31929] pm1 cib: debug:
cib_file_write_with_digest: Writing CIB to disk
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_file_write_with_digest: Wrote version 0.19.0 of the CIB to disk
(digest: 19aae215a8e3f0a0ed1116cf6276bc3f)
Oct 12 18:41:28 [31933] pm1 pengine: info: process_pe_message:
Input has not changed since last time, not saving to disk
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:28 [31933] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:28 [31933] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:28 [31933] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:28 [31933] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:41:28 [31933] pm1 pengine: info: short_print:
Stopped: [ pm1 pm2 ]
Oct 12 18:41:28 [31933] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:41:28 [31933] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:28 [31933] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:0
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:1
Oct 12 18:41:28 [31933] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:41:28 [31933] pm1 pengine: debug: master_color: pgsql:0
master score: -1
Oct 12 18:41:28 [31933] pm1 pengine: debug: master_color: pgsql:1
master score: -1
Oct 12 18:41:28 [31933] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:41:28 [31933] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:41:28 [31933] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:41:28 [31933] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing pgsql:0 on pm1 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-master on pm1 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-rep on pm1 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing pgsql:1 on pm2 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-master on pm2 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
native_create_probe: Probing vip-rep on pm2 (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm1
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm2
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm1
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm2
Oct 12 18:41:28 [31933] pm1 pengine: notice: LogActions: Start
pgsql:0 (pm1)
Oct 12 18:41:28 [31933] pm1 pengine: notice: LogActions: Start
pgsql:1 (pm2)
Oct 12 18:41:28 [31933] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: notice: process_pe_message:
Calculated Transition 2: /var/lib/pacemaker/pengine/pe-input-109.bz2
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:41:28 [31934] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:41:28 [31934] pm1 crmd: debug: unpack_graph: Unpacked
transition 2: 21 actions in 21 synapses
Oct 12 18:41:28 [31934] pm1 crmd: info: do_te_invoke:
Processing graph 2 (ref=pe_calc-dc-1444664488-14) derived from
/var/lib/pacemaker/pengine/pe-input-109.bz2
Oct 12 18:41:28 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 4: monitor pgsql:0_monitor_0 on pm1 (local)
Oct 12 18:41:28 [31931] pm1 lrmd: info:
process_lrmd_get_rsc_info: Resource 'pgsql' not found (0 active
resources)
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_info operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=0, reply=0, notify=0, exit=4203648
Oct 12 18:41:28 [31929] pm1 cib: debug:
cib_file_write_with_digest: Wrote digest
19aae215a8e3f0a0ed1116cf6276bc3f to disk
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_file_write_with_digest: Reading cluster configuration file
/var/lib/pacemaker/cib/cib.mtyZXp (digest:
/var/lib/pacemaker/cib/cib.YdSDcD)
Oct 12 18:41:28 [31929] pm1 cib: debug:
cib_file_write_with_digest: Activating /var/lib/pacemaker/cib/cib.mtyZXp
Oct 12 18:41:28 [31931] pm1 lrmd: info:
process_lrmd_get_rsc_info: Resource 'pgsql:0' not found (0 active
resources)
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_info operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=0, reply=0, notify=0, exit=4203648
Oct 12 18:41:28 [31931] pm1 lrmd: info:
process_lrmd_rsc_register: Added 'pgsql' to the rsc list (1 active
resources)
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_register operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=0, reply=1, notify=1, exit=4203648
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_info operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=0, reply=0, notify=0, exit=4203648
Oct 12 18:41:28 [31934] pm1 crmd: info: do_lrm_rsc_op:
Performing key=4:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213 op=pgsql_monitor_0
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=6, reply=1, notify=0, exit=4203648
Oct 12 18:41:28 [31931] pm1 lrmd: debug: log_execute: executing
- rsc:pgsql action:monitor call_id:6
Oct 12 18:41:28 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 8: monitor pgsql:1_monitor_0 on pm2
Oct 12 18:41:28 [31931] pm1 lrmd: debug: child_waitpid:
wait(31938) = 0: Resource temporarily unavailable (11)
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 17 fired and confirmed
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
2 (Complete=0, Pending=2, Fired=3, Skipped=0, Incomplete=18,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): In-progress
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 18 fired and confirmed
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 15 fired and confirmed
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
2 (Complete=1, Pending=2, Fired=2, Skipped=0, Incomplete=16,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): In-progress
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
2 (Complete=3, Pending=2, Fired=0, Skipped=0, Incomplete=16,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): In-progress
Oct 12 18:41:28 [31972] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:28 [31972] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-31913-31972-24-header
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-31913-31972-24-header
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-31913-31972-24-header
Oct 12 18:41:28 [31972] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-31972-24-header
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-31972-24-header
Oct 12 18:41:28 [31972] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-31972-24-header
Oct 12 18:41:28 [31972] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
pgsql(pgsql)[31938]: 2015/10/12_18:41:28 INFO: Don't check
/var/lib/pgsql/9.3/data/ during probe
pgsql(pgsql)[31938]: 2015/10/12_18:41:28 INFO: PostgreSQL is down
Oct 12 18:41:28 [31931] pm1 lrmd: debug: operation_finished:
pgsql_monitor_0:31938 - exited with rc=7
Oct 12 18:41:28 [31931] pm1 lrmd: debug: operation_finished:
pgsql_monitor_0:31938:stderr [ -- empty -- ]
Oct 12 18:41:28 [31931] pm1 lrmd: debug: operation_finished:
pgsql_monitor_0:31938:stdout [ -- empty -- ]
Oct 12 18:41:28 [31931] pm1 lrmd: debug: log_finished: finished
- rsc:pgsql action:monitor call_id:6 pid:31938 exit-code:7
exec-time:88ms queue-time:0ms
Oct 12 18:41:28 [31934] pm1 crmd: debug:
create_operation_update: do_update_resource: Updating resource pgsql
after monitor op complete (interval=0)
Oct 12 18:41:28 [32057] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:28 [32057] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-31913-32057-24-header
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-31913-32057-24-header
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-31913-32057-24-header
Oct 12 18:41:28 [32057] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-32057-24-header
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-32057-24-header
Oct 12 18:41:28 [32057] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-32057-24-header
Oct 12 18:41:28 [32057] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:28 [31934] pm1 crmd: info: action_synced_wait:
Managed pgsql_meta-data_0 process 32023 exited with rc=0
Oct 12 18:41:28 [31934] pm1 crmd: notice: process_lrm_event:
Operation pgsql_monitor_0: not running (node=pm1, call=6, rc=7,
cib-update=36, confirmed=true)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
update_history_cache: Updating history for 'pgsql' with monitor op
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/36)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.7 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.8 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=8
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']: @crm-debug-origin=do_update_resource
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources:
<lrm_resource id="pgsql" type="pgsql" class="ocf" provider="heartbeat"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
<lrm_rsc_op id="pgsql_last_0" operation_key="pgsql_monitor_0"
operation="monitor" crm-debug-origin="do_update_resource"
crm_feature_set="3.0.10"
transition-key="4:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
transition-magic="0:7;4:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
on_node="pm1" call-id="6" rc-code="7" op-status="0" interval="0"
last-run="1444664488" last-rc-change="14446
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
</lrm_resource>
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/36, version=0.19.8)
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.8 to 0.19.7
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.7 -> 0.19.8 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info: match_graph_event:
Action pgsql_monitor_0 (4) confirmed on pm1 (rc=7)
Oct 12 18:41:28 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 5: monitor vip-master_monitor_0 on pm1 (local)
Oct 12 18:41:28 [31931] pm1 lrmd: info:
process_lrmd_get_rsc_info: Resource 'vip-master' not found (1 active
resources)
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_info operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=0, reply=0, notify=0, exit=4203648
Oct 12 18:41:28 [31931] pm1 lrmd: info:
process_lrmd_rsc_register: Added 'vip-master' to the rsc list (2
active resources)
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_register operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=0, reply=1, notify=1, exit=4203648
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_info operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=0, reply=0, notify=0, exit=4203648
Oct 12 18:41:28 [31934] pm1 crmd: info: do_lrm_rsc_op:
Performing key=5:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213
op=vip-master_monitor_0
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=10, reply=1, notify=0, exit=4203648
Oct 12 18:41:28 [31931] pm1 lrmd: debug: log_execute: executing
- rsc:vip-master action:monitor call_id:10
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
2 (Complete=4, Pending=2, Fired=1, Skipped=0, Incomplete=15,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): In-progress
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.8 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.9 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=9
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='2']: @crm-debug-origin=do_update_resource
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources:
<lrm_resource id="pgsql" type="pgsql" class="ocf" provider="heartbeat"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
<lrm_rsc_op id="pgsql_last_0" operation_key="pgsql_monitor_0"
operation="monitor" crm-debug-origin="do_update_resource"
crm_feature_set="3.0.10"
transition-key="8:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
transition-magic="0:7;8:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
on_node="pm2" call-id="6" rc-code="7" op-status="0" interval="0"
last-run="1444664488" last-rc-change="14446
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
</lrm_resource>
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.9 to 0.19.8
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.8 -> 0.19.9 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info: match_graph_event:
Action pgsql_monitor_0 (8) confirmed on pm2 (rc=7)
Oct 12 18:41:28 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 9: monitor vip-master_monitor_0 on pm2
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
2 (Complete=5, Pending=2, Fired=1, Skipped=0, Incomplete=14,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): In-progress
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/crmd/11, version=0.19.9)
Oct 12 18:41:28 [31931] pm1 lrmd: debug: operation_finished:
vip-master_monitor_0:32059 - exited with rc=7
Oct 12 18:41:28 [31931] pm1 lrmd: debug: operation_finished:
vip-master_monitor_0:32059:stderr [ -- empty -- ]
Oct 12 18:41:28 [31931] pm1 lrmd: debug: operation_finished:
vip-master_monitor_0:32059:stdout [ -- empty -- ]
Oct 12 18:41:28 [31931] pm1 lrmd: debug: log_finished: finished
- rsc:vip-master action:monitor call_id:10 pid:32059 exit-code:7
exec-time:43ms queue-time:0ms
Oct 12 18:41:28 [31934] pm1 crmd: debug:
create_operation_update: do_update_resource: Updating resource
vip-master after monitor op complete (interval=0)
Oct 12 18:41:28 [31934] pm1 crmd: info: action_synced_wait:
Managed IPaddr2_meta-data_0 process 32105 exited with rc=0
Oct 12 18:41:28 [31934] pm1 crmd: notice: process_lrm_event:
Operation vip-master_monitor_0: not running (node=pm1, call=10, rc=7,
cib-update=37, confirmed=true)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
update_history_cache: Updating history for 'vip-master' with monitor op
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/37)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.9 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.10 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=10
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources:
<lrm_resource id="vip-master" type="IPaddr2" class="ocf"
provider="heartbeat"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
<lrm_rsc_op id="vip-master_last_0" operation_key="vip-master_monitor_0"
operation="monitor" crm-debug-origin="do_update_resource"
crm_feature_set="3.0.10"
transition-key="5:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
transition-magic="0:7;5:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
on_node="pm1" call-id="10" rc-code="7" op-status="0" interval="0"
last-run="1444664488" last-rc-ch
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
</lrm_resource>
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/37, version=0.19.10)
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.10 to 0.19.9
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.9 -> 0.19.10 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info: match_graph_event:
Action vip-master_monitor_0 (5) confirmed on pm1 (rc=7)
Oct 12 18:41:28 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 6: monitor vip-rep_monitor_0 on pm1 (local)
Oct 12 18:41:28 [31931] pm1 lrmd: info:
process_lrmd_get_rsc_info: Resource 'vip-rep' not found (2 active
resources)
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_info operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=0, reply=0, notify=0, exit=4203648
Oct 12 18:41:28 [31931] pm1 lrmd: info:
process_lrmd_rsc_register: Added 'vip-rep' to the rsc list (3 active
resources)
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_register operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=0, reply=1, notify=1, exit=4203648
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_info operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=0, reply=0, notify=0, exit=4203648
Oct 12 18:41:28 [31934] pm1 crmd: info: do_lrm_rsc_op:
Performing key=6:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213
op=vip-rep_monitor_0
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=14, reply=1, notify=0, exit=4203648
Oct 12 18:41:28 [31931] pm1 lrmd: debug: log_execute: executing
- rsc:vip-rep action:monitor call_id:14
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
2 (Complete=6, Pending=2, Fired=1, Skipped=0, Incomplete=13,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): In-progress
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.10 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.11 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=11
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources:
<lrm_resource id="vip-master" type="IPaddr2" class="ocf"
provider="heartbeat"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
<lrm_rsc_op id="vip-master_last_0" operation_key="vip-master_monitor_0"
operation="monitor" crm-debug-origin="do_update_resource"
crm_feature_set="3.0.10"
transition-key="9:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
transition-magic="0:7;9:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
on_node="pm2" call-id="10" rc-code="7" op-status="0" interval="0"
last-run="1444664488" last-rc-ch
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
</lrm_resource>
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/crmd/12, version=0.19.11)
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.11 to 0.19.10
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.10 -> 0.19.11 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info: match_graph_event:
Action vip-master_monitor_0 (9) confirmed on pm2 (rc=7)
Oct 12 18:41:28 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 10: monitor vip-rep_monitor_0 on pm2
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
2 (Complete=7, Pending=2, Fired=1, Skipped=0, Incomplete=12,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): In-progress
Oct 12 18:41:28 [31931] pm1 lrmd: debug: operation_finished:
vip-rep_monitor_0:32109 - exited with rc=7
Oct 12 18:41:28 [31931] pm1 lrmd: debug: operation_finished:
vip-rep_monitor_0:32109:stderr [ -- empty -- ]
Oct 12 18:41:28 [31931] pm1 lrmd: debug: operation_finished:
vip-rep_monitor_0:32109:stdout [ -- empty -- ]
Oct 12 18:41:28 [31931] pm1 lrmd: debug: log_finished: finished
- rsc:vip-rep action:monitor call_id:14 pid:32109 exit-code:7
exec-time:38ms queue-time:0ms
Oct 12 18:41:28 [31934] pm1 crmd: debug:
create_operation_update: do_update_resource: Updating resource
vip-rep after monitor op complete (interval=0)
Oct 12 18:41:28 [31934] pm1 crmd: notice: process_lrm_event:
Operation vip-rep_monitor_0: not running (node=pm1, call=14, rc=7,
cib-update=38, confirmed=true)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
update_history_cache: Updating history for 'vip-rep' with monitor op
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/38)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.11 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.12 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=12
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources:
<lrm_resource id="vip-rep" type="IPaddr2" class="ocf" provider="heartbeat"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
<lrm_rsc_op id="vip-rep_last_0" operation_key="vip-rep_monitor_0"
operation="monitor" crm-debug-origin="do_update_resource"
crm_feature_set="3.0.10"
transition-key="6:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
transition-magic="0:7;6:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
on_node="pm1" call-id="14" rc-code="7" op-status="0" interval="0"
last-run="1444664488" last-rc-change="
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
</lrm_resource>
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.12 to 0.19.11
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/38, version=0.19.12)
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.11 -> 0.19.12 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info: match_graph_event:
Action vip-rep_monitor_0 (6) confirmed on pm1 (rc=7)
Oct 12 18:41:28 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 3: probe_complete probe_complete-pm1 on pm1 (local) -
no waiting
Oct 12 18:41:28 [31934] pm1 crmd: debug:
attrd_update_delegate: Sent update: probe_complete=true for pm1
Oct 12 18:41:28 [31934] pm1 crmd: info: te_rsc_command: Action
3 confirmed - no wait
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
2 (Complete=8, Pending=1, Fired=1, Skipped=0, Incomplete=11,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): In-progress
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
2 (Complete=9, Pending=1, Fired=0, Skipped=0, Incomplete=11,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): In-progress
Oct 12 18:41:28 [31932] pm1 attrd: debug:
attrd_client_update: Broadcasting probe_complete[pm1] = true (writer)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_peer_update:
Setting probe_complete[pm1]: (null) -> true from pm1
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[probe_complete]=true (1 1 1 pm1)
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Sent update 5 with 1 changes for probe_complete, id=<n/a>, set=(null)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/5)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.12 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.13 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=13
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']:
<nvpair id="status-1-probe_complete" name="probe_complete" value="true"/>
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.13 to 0.19.12
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.12 -> 0.19.13 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
update_abort_priority: Abort priority upgraded from 0 to 1000000
Oct 12 18:41:28 [31934] pm1 crmd: debug:
update_abort_priority: Abort action done superseded by restart:
Transient attribute change
Oct 12 18:41:28 [31934] pm1 crmd: notice:
abort_transition_graph: Transition aborted by
status-1-probe_complete, probe_complete=true: Transient attribute change
(create cib=0.19.13, source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'],
0)
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
2 (Complete=9, Pending=1, Fired=0, Skipped=0, Incomplete=11,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): In-progress
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/5, version=0.19.13)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 5 for probe_complete: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 5 for probe_complete[pm1]=true: OK (0)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.13 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.14 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=14
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources:
<lrm_resource id="vip-rep" type="IPaddr2" class="ocf" provider="heartbeat"/>
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
<lrm_rsc_op id="vip-rep_last_0" operation_key="vip-rep_monitor_0"
operation="monitor" crm-debug-origin="do_update_resource"
crm_feature_set="3.0.10"
transition-key="10:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
transition-magic="0:7;10:2:7:b020e8a2-0497-404a-b66a-e629cc7d3213"
on_node="pm2" call-id="14" rc-code="7" op-status="0" interval="0"
last-run="1444664488" last-rc-change
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
</lrm_resource>
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.14 to 0.19.13
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.13 -> 0.19.14 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info: match_graph_event:
Action vip-rep_monitor_0 (10) confirmed on pm2 (rc=7)
Oct 12 18:41:28 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 7: probe_complete probe_complete-pm2 on pm2 - no waiting
Oct 12 18:41:28 [31934] pm1 crmd: info: te_rsc_command: Action
7 confirmed - no wait
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 2 fired and confirmed
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_peer_update:
Setting probe_complete[pm2]: (null) -> true from pm2
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[probe_complete]=true (1 1 1 pm1)
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm2[probe_complete]=true (2 2 2 pm2)
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Sent update 6 with 2 changes for probe_complete, id=<n/a>, set=(null)
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
2 (Complete=10, Pending=0, Fired=2, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): In-progress
Oct 12 18:41:28 [31934] pm1 crmd: notice: run_graph: Transition
2 (Complete=12, Pending=0, Fired=0, Skipped=2, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-input-109.bz2): Stopped
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_graph_trigger:
Transition 2 is now complete
Oct 12 18:41:28 [31934] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:41:28 [31934] pm1 crmd: debug: notify_crmd:
Transition 2 status: restart - Transient attribute change
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:41:28 [31934] pm1 crmd: info:
do_state_transition: State transition S_TRANSITION_ENGINE ->
S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:41:28 [31934] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_pe_invoke: Query
39: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/crmd/13, version=0.19.14)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/6)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=39,
ref=pe_calc-dc-1444664488-23, seq=172, quorate=1
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:28 [31933] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:28 [31933] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:28 [31933] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:28 [31933] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:41:28 [31933] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:0
Oct 12 18:41:28 [31933] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:41:28 [31933] pm1 pengine: info: short_print:
Stopped: [ pm1 pm2 ]
Oct 12 18:41:28 [31933] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:41:28 [31933] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:28 [31933] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:0
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:1
Oct 12 18:41:28 [31933] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:41:28 [31933] pm1 pengine: debug: master_color: pgsql:0
master score: -1
Oct 12 18:41:28 [31933] pm1 pengine: debug: master_color: pgsql:1
master score: -1
Oct 12 18:41:28 [31933] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:41:28 [31933] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:41:28 [31933] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:41:28 [31933] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:41:28 [31933] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm1
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm2
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm1
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm2
Oct 12 18:41:28 [31933] pm1 pengine: notice: LogActions: Start
pgsql:0 (pm1)
Oct 12 18:41:28 [31933] pm1 pengine: notice: LogActions: Start
pgsql:1 (pm2)
Oct 12 18:41:28 [31933] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: notice: process_pe_message:
Calculated Transition 3: /var/lib/pacemaker/pengine/pe-input-110.bz2
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:41:28 [31934] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:41:28 [31934] pm1 crmd: debug: unpack_graph: Unpacked
transition 3: 13 actions in 13 synapses
Oct 12 18:41:28 [31934] pm1 crmd: info: do_te_invoke:
Processing graph 3 (ref=pe_calc-dc-1444664488-23) derived from
/var/lib/pacemaker/pengine/pe-input-110.bz2
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 11 fired and confirmed
Oct 12 18:41:28 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 4: probe_complete probe_complete-pm2 on pm2 - no waiting
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.14 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.15 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=15
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']:
<nvpair id="status-2-probe_complete" name="probe_complete" value="true"/>
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/6, version=0.19.15)
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.15 to 0.19.14
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 6 for probe_complete: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 6 for probe_complete[pm1]=true: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 6 for probe_complete[pm2]=true: OK (0)
Oct 12 18:41:28 [31934] pm1 crmd: info: te_rsc_command: Action
4 confirmed - no wait
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
3 (Complete=0, Pending=0, Fired=2, Skipped=0, Incomplete=11,
Source=/var/lib/pacemaker/pengine/pe-input-110.bz2): In-progress
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.14 -> 0.19.15 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
update_abort_priority: Abort priority upgraded from 0 to 1000000
Oct 12 18:41:28 [31934] pm1 crmd: debug:
update_abort_priority: Abort action done superseded by restart:
Transient attribute change
Oct 12 18:41:28 [31934] pm1 crmd: notice:
abort_transition_graph: Transition aborted by
status-2-probe_complete, probe_complete=true: Transient attribute change
(create cib=0.19.15, source=te_update_diff:391,
path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'],
0)
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 12 fired and confirmed
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 9 fired and confirmed
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
3 (Complete=2, Pending=0, Fired=2, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-input-110.bz2): In-progress
Oct 12 18:41:28 [31934] pm1 crmd: notice: run_graph: Transition
3 (Complete=4, Pending=0, Fired=0, Skipped=2, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-input-110.bz2): Stopped
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_graph_trigger:
Transition 3 is now complete
Oct 12 18:41:28 [31934] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:41:28 [31934] pm1 crmd: debug: notify_crmd:
Transition 3 status: restart - Transient attribute change
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:41:28 [31934] pm1 crmd: info:
do_state_transition: State transition S_TRANSITION_ENGINE ->
S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:41:28 [31934] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_pe_invoke: Query
40: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31934] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=40,
ref=pe_calc-dc-1444664488-25, seq=172, quorate=1
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:28 [31933] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:28 [31933] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:28 [31933] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:28 [31933] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:28 [31933] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:41:28 [31933] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:0
Oct 12 18:41:28 [31933] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:41:28 [31933] pm1 pengine: info: short_print:
Stopped: [ pm1 pm2 ]
Oct 12 18:41:28 [31933] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:41:28 [31933] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:28 [31933] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:0
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:1
Oct 12 18:41:28 [31933] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:41:28 [31933] pm1 pengine: debug: master_color: pgsql:0
master score: -1
Oct 12 18:41:28 [31933] pm1 pengine: debug: master_color: pgsql:1
master score: -1
Oct 12 18:41:28 [31933] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:41:28 [31933] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:41:28 [31933] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:41:28 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:41:28 [31933] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:41:28 [31933] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm1
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm2
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm1
Oct 12 18:41:28 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm2
Oct 12 18:41:28 [31933] pm1 pengine: notice: LogActions: Start
pgsql:0 (pm1)
Oct 12 18:41:28 [31933] pm1 pengine: notice: LogActions: Start
pgsql:1 (pm2)
Oct 12 18:41:28 [31933] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. (null))
Oct 12 18:41:28 [31933] pm1 pengine: notice: process_pe_message:
Calculated Transition 4: /var/lib/pacemaker/pengine/pe-input-111.bz2
Oct 12 18:41:28 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:41:28 [31934] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:41:28 [31934] pm1 crmd: debug: unpack_graph: Unpacked
transition 4: 12 actions in 12 synapses
Oct 12 18:41:28 [31934] pm1 crmd: info: do_te_invoke:
Processing graph 4 (ref=pe_calc-dc-1444664488-25) derived from
/var/lib/pacemaker/pengine/pe-input-111.bz2
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 11 fired and confirmed
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=0, Pending=0, Fired=1, Skipped=0, Incomplete=11,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 12 fired and confirmed
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 9 fired and confirmed
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=1, Pending=0, Fired=2, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:28 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 5: start pgsql_start_0 on pm1 (local)
Oct 12 18:41:28 [31934] pm1 crmd: debug: do_lrm_rsc_op: Stopped
0 recurring operations in preparation for pgsql_start_0
Oct 12 18:41:28 [31934] pm1 crmd: info: do_lrm_rsc_op:
Performing key=5:4:0:b020e8a2-0497-404a-b66a-e629cc7d3213 op=pgsql_start_0
Oct 12 18:41:28 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=15, reply=1, notify=0, exit=4203648
Oct 12 18:41:28 [31931] pm1 lrmd: info: log_execute: executing
- rsc:pgsql action:start call_id:15
Oct 12 18:41:28 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 7: start pgsql:1_start_0 on pm2
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=3, Pending=2, Fired=2, Skipped=0, Incomplete=7,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:28 [32189] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:28 [32189] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-31913-32189-24-header
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-31913-32189-24-header
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-31913-32189-24-header
Oct 12 18:41:28 [32189] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-32189-24-header
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-32189-24-header
Oct 12 18:41:28 [32189] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-32189-24-header
Oct 12 18:41:28 [32189] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:28 [32245] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf21dc0 for uid=0 gid=0 pid=32245
id=c08b5110-5471-4bc6-b507-6937d5352143
Oct 12 18:41:28 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32245-14)
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32245]
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32245] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32245] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32245] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32245] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [32245] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:28 [32245] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:28 [32245] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:28 [32245] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:28 [32245] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:28 [32245] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:28 [32245] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:28 [32245] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:28 [32245] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:28 [32245] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:41:28 [32245] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:0
Oct 12 18:41:28 [32245] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32245-14)
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32245-14) state:2
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-31929-32245-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-31929-32245-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-31929-32245-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf21dc0 for uid=0 gid=0 pid=32247
id=89d387a5-d0cf-4ff9-8b3c-b2985067e8a9
Oct 12 18:41:28 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32247-14)
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32247]
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:28 [32247] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_process_xpath:
cib_query:
//cib/status//node_state[@id='1']//transient_attributes//nvpair[@name='pgsql-status']
does not exist
Oct 12 18:41:28 [32247] pm1 crm_attribute: info: main: Read
pgsql-status=<null>
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32247-14-header
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32247-14-header
Oct 12 18:41:28 [32247] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32247-14-header
Oct 12 18:41:28 [32247] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32247-14)
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32247-14) state:2
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32247-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32247-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32247-14-header
pgsql(pgsql)[32155]: 2015/10/12_18:41:28 INFO: Changing pgsql-status
on pm1 : ->STOP.
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf21dc0 for uid=0 gid=0 pid=32252
id=6a183548-dbea-40b0-a868-4d2a8d04fec6
Oct 12 18:41:28 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32252-14)
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32252]
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:28 [32252] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:28 [32252] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:41:28 [31932] pm1 attrd: debug: crm_client_new:
Connecting 0xbcd700 for uid=0 gid=0 pid=32252
id=8c5945d1-ed4d-46ff-9641-5806a64feace
Oct 12 18:41:28 [31932] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (31932-32252-11)
Oct 12 18:41:28 [31932] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [32252]
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-status[pm1] = STOP (writer)
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-status=STOP for pm1
Oct 12 18:41:28 [32252] pm1 crm_attribute: info: main: Update
pgsql-status=STOP sent via attrd
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32252-14-header
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32252-14-header
Oct 12 18:41:28 [32252] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32252-14-header
Oct 12 18:41:28 [32252] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32252-14)
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32252-14) state:2
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32252-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32252-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32252-14-header
Oct 12 18:41:28 [31932] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31932-32252-11)
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31932-32252-11) state:2
Oct 12 18:41:28 [31932] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-31932-32252-11-header
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-31932-32252-11-header
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-31932-32252-11-header
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_peer_update:
Setting pgsql-status[pm1]: (null) -> STOP from pm1
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[pgsql-status]=STOP (1 1 1 pm1)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/7)
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Sent update 7 with 1 changes for pgsql-status, id=<n/a>, set=(null)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.15 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.16 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=16
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']:
<nvpair id="status-1-pgsql-status" name="pgsql-status" value="STOP"/>
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.15 -> 0.19.16 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: debug:
update_abort_priority: Abort priority upgraded from 0 to 1000000
Oct 12 18:41:28 [31934] pm1 crmd: debug:
update_abort_priority: Abort action done superseded by restart:
Transient attribute change
Oct 12 18:41:28 [31934] pm1 crmd: notice:
abort_transition_graph: Transition aborted by status-1-pgsql-status,
pgsql-status=STOP: Transient attribute change (create cib=0.19.16,
source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'],
0)
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=3, Pending=2, Fired=0, Skipped=0, Incomplete=7,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.16 to 0.19.15
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/7, version=0.19.16)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 7 for pgsql-status: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 7 for pgsql-status[pm1]=STOP: OK (0)
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf21dc0 for uid=0 gid=0 pid=32253
id=aeb6d59c-6ef7-43e2-93ea-b47efd8b0592
Oct 12 18:41:28 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32253-14)
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32253]
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:28 [32253] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:28 [32253] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:41:28 [31932] pm1 attrd: debug: crm_client_new:
Connecting 0xbcd700 for uid=0 gid=0 pid=32253
id=501eafdf-bba4-478d-a4be-01928f0533db
Oct 12 18:41:28 [31932] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (31932-32253-11)
Oct 12 18:41:28 [31932] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [32253]
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-master-baseline[pm1] = (null)
(writer)
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-master-baseline=(null) for pm1
Oct 12 18:41:28 [32253] pm1 crm_attribute: info: main: Update
pgsql-master-baseline=<none> sent via attrd
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32253-14-header
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32253-14-header
Oct 12 18:41:28 [32253] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32253-14-header
Oct 12 18:41:28 [32253] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32253-14)
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32253-14) state:2
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32253-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32253-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32253-14-header
Oct 12 18:41:28 [31932] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31932-32253-11)
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31932-32253-11) state:2
Oct 12 18:41:28 [31932] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-31932-32253-11-header
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-31932-32253-11-header
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-31932-32253-11-header
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_peer_update:
Setting pgsql-status[pm2]: (null) -> STOP from pm2
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[pgsql-status]=STOP (1 1 1 pm1)
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm2[pgsql-status]=STOP (2 2 2 pm2)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/8)
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Sent update 8 with 2 changes for pgsql-status, id=<n/a>, set=(null)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.16 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.17 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=17
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']:
<nvpair id="status-2-pgsql-status" name="pgsql-status" value="STOP"/>
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.17 to 0.19.16
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.16 -> 0.19.17 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info:
abort_transition_graph: Transition aborted by status-2-pgsql-status,
pgsql-status=STOP: Transient attribute change (create cib=0.19.17,
source=te_update_diff:391,
path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'],
0)
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=3, Pending=2, Fired=0, Skipped=0, Incomplete=7,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/8, version=0.19.17)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 8 for pgsql-status: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 8 for pgsql-status[pm1]=STOP: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 8 for pgsql-status[pm2]=STOP: OK (0)
Oct 12 18:41:28 [32255] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:28 [32255] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-31913-32255-24-header
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-31913-32255-24-header
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-31913-32255-24-header
Oct 12 18:41:28 [32255] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-32255-24-header
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-32255-24-header
Oct 12 18:41:28 [32255] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-32255-24-header
Oct 12 18:41:28 [32255] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xc7d9f0 for uid=0 gid=0 pid=32257
id=3e3b3283-5d18-40f7-8104-fd6d35647337
Oct 12 18:41:28 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32257-14)
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32257]
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:28 [32257] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:28 [32257] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:41:28 [31932] pm1 attrd: debug: crm_client_new:
Connecting 0xbcd700 for uid=0 gid=0 pid=32257
id=3f6c74d2-7521-4033-939f-c01954078377
Oct 12 18:41:28 [31932] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (31932-32257-11)
Oct 12 18:41:28 [31932] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [32257]
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug:
attrd_client_update: Broadcasting master-pgsql[pm1] = -INFINITY (writer)
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: master-pgsql=-INFINITY for pm1
Oct 12 18:41:28 [32257] pm1 crm_attribute: info: main: Update
master-pgsql=-INFINITY sent via attrd
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32257-14-header
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32257-14-header
Oct 12 18:41:28 [32257] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32257-14-header
Oct 12 18:41:28 [32257] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32257-14)
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32257-14) state:2
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32257-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32257-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32257-14-header
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_peer_update:
Setting master-pgsql[pm1]: (null) -> -INFINITY from pm1
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[master-pgsql]=-INFINITY (1 1 1 pm1)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/9)
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Sent update 9 with 1 changes for master-pgsql, id=<n/a>, set=(null)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31932] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31932-32257-11)
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31932-32257-11) state:2
Oct 12 18:41:28 [31932] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-31932-32257-11-header
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-31932-32257-11-header
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-31932-32257-11-header
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.17 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.18 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=18
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']:
<nvpair id="status-1-master-pgsql" name="master-pgsql" value="-INFINITY"/>
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/9, version=0.19.18)
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.18 to 0.19.17
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.17 -> 0.19.18 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info:
abort_transition_graph: Transition aborted by status-1-master-pgsql,
master-pgsql=-INFINITY: Transient attribute change (create cib=0.19.18,
source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'],
0)
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=3, Pending=2, Fired=0, Skipped=0, Incomplete=7,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 9 for master-pgsql: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 9 for master-pgsql[pm1]=-INFINITY: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_peer_update:
Setting master-pgsql[pm2]: (null) -> -INFINITY from pm2
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[master-pgsql]=-INFINITY (1 1 1 pm1)
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm2[master-pgsql]=-INFINITY (2 2 2 pm2)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/10)
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Sent update 10 with 2 changes for master-pgsql, id=<n/a>, set=(null)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.18 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.19 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=19
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']:
<nvpair id="status-2-master-pgsql" name="master-pgsql" value="-INFINITY"/>
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.19 to 0.19.18
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.18 -> 0.19.19 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info:
abort_transition_graph: Transition aborted by status-2-master-pgsql,
master-pgsql=-INFINITY: Transient attribute change (create cib=0.19.19,
source=te_update_diff:391,
path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'],
0)
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=3, Pending=2, Fired=0, Skipped=0, Incomplete=7,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/10, version=0.19.19)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 10 for master-pgsql: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 10 for master-pgsql[pm1]=-INFINITY: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 10 for master-pgsql[pm2]=-INFINITY: OK (0)
pgsql(pgsql)[32155]: 2015/10/12_18:41:28 DEBUG: Created
recovery.conf. host=10.200.200.225, user=postgres
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xc7dbd0 for uid=0 gid=0 pid=32272
id=d5375305-38ab-4157-9f11-83b3b07ff556
Oct 12 18:41:28 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32272-14)
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32272]
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:28 [32272] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:28 [32272] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:41:28 [31932] pm1 attrd: debug: crm_client_new:
Connecting 0xbcd700 for uid=0 gid=0 pid=32272
id=1d2e0745-1ffb-49e1-9793-51b117cb123e
Oct 12 18:41:28 [31932] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (31932-32272-11)
Oct 12 18:41:28 [31932] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [32272]
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:28 [31932] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-xlog-loc[pm1] = (null) (writer)
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-xlog-loc=(null) for pm1
Oct 12 18:41:28 [32272] pm1 crm_attribute: info: main: Update
pgsql-xlog-loc=<none> sent via attrd
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32272-14-header
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32272-14-header
Oct 12 18:41:28 [32272] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32272-14-header
Oct 12 18:41:28 [32272] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:28 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32272-14)
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32272-14) state:2
Oct 12 18:41:28 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32272-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32272-14-header
Oct 12 18:41:28 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32272-14-header
Oct 12 18:41:28 [31932] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31932-32272-11)
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31932-32272-11) state:2
Oct 12 18:41:28 [31932] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-31932-32272-11-header
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-31932-32272-11-header
Oct 12 18:41:28 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-31932-32272-11-header
pgsql(pgsql)[32155]: 2015/10/12_18:41:28 INFO: Set all nodes into
async mode.
pgsql(pgsql)[32155]: 2015/10/12_18:41:28 INFO: server starting
pgsql(pgsql)[32155]: 2015/10/12_18:41:28 INFO: PostgreSQL start
command sent.
pgsql(pgsql)[32155]: 2015/10/12_18:41:28 WARNING: Can't get
PostgreSQL recovery status. rc=2
pgsql(pgsql)[32155]: 2015/10/12_18:41:28 WARNING: Connection error
(connection to the server went bad and the session was not interactive)
occurred while executing the psql command.
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_peer_update:
Setting pgsql-status[pm2]: STOP -> HS:alone from pm2
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[pgsql-status]=STOP (1 1 1 pm1)
Oct 12 18:41:28 [31932] pm1 attrd: debug: write_attribute:
Update: pm2[pgsql-status]=HS:alone (2 2 2 pm2)
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/11)
Oct 12 18:41:28 [31932] pm1 attrd: info: write_attribute:
Sent update 11 with 2 changes for pgsql-status, id=<n/a>, set=(null)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.19 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.20 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=20
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-pgsql-status']:
@value=HS:alone
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.20 to 0.19.19
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/11, version=0.19.20)
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.19 -> 0.19.20 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info:
abort_transition_graph: Transition aborted by status-2-pgsql-status,
pgsql-status=HS:alone: Transient attribute change (modify cib=0.19.20,
source=te_update_diff:391,
path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']/nvpair[@id='status-2-pgsql-status'],
0)
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=3, Pending=2, Fired=0, Skipped=0, Incomplete=7,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 11 for pgsql-status: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 11 for pgsql-status[pm1]=STOP: OK (0)
Oct 12 18:41:28 [31932] pm1 attrd: info: attrd_cib_callback:
Update 11 for pgsql-status[pm2]=HS:alone: OK (0)
Oct 12 18:41:28 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.20 2
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.21 (null)
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=21
Oct 12 18:41:28 [31929] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='pgsql']/lrm_rsc_op[@id='pgsql_last_0']:
@operation_key=pgsql_start_0, @operation=start,
@transition-key=7:4:0:b020e8a2-0497-404a-b66a-e629cc7d3213,
@transition-magic=0:0;7:4:0:b020e8a2-0497-404a-b66a-e629cc7d3213,
@call-id=15, @rc-code=0, @exec-time=342
Oct 12 18:41:28 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/crmd/14, version=0.19.21)
Oct 12 18:41:28 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.20 -> 0.19.21 (S_TRANSITION_ENGINE)
Oct 12 18:41:28 [31934] pm1 crmd: info: match_graph_event:
Action pgsql_start_0 (7) confirmed on pm2 (rc=0)
Oct 12 18:41:28 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=4, Pending=1, Fired=0, Skipped=0, Incomplete=7,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:28 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.21 to 0.19.20
pgsql(pgsql)[32155]: 2015/10/12_18:41:29 DEBUG: PostgreSQL still
hasn't started yet. Waiting...
pgsql(pgsql)[32155]: 2015/10/12_18:41:29 DEBUG: PostgreSQL is running
as a hot standby.
pgsql(pgsql)[32155]: 2015/10/12_18:41:29 INFO: PostgreSQL is started.
Oct 12 18:41:29 [32389] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:41:29 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf235d0 for uid=0 gid=0 pid=32389
id=431f6f0b-cfc3-4785-b823-7f2a822724a0
Oct 12 18:41:29 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32389-14)
Oct 12 18:41:29 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32389]
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32389] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32389] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32389] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32389] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:29 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:29 [32389] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:29 [32389] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:29 [32389] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:29 [32389] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:29 [32389] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:29 [32389] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:29 [32389] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:29 [32389] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:29 [32389] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:29 [32389] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:41:29 [32389] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:41:29 [32389] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:41:29 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32389-14)
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32389-14) state:2
Oct 12 18:41:29 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-31929-32389-14-header
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-31929-32389-14-header
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-31929-32389-14-header
Oct 12 18:41:29 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf235d0 for uid=0 gid=0 pid=32391
id=96056d88-ab7f-4ade-a147-8f58f9de672f
Oct 12 18:41:29 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32391-14)
Oct 12 18:41:29 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32391]
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:29 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:29 [32391] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:29 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:29 [31929] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/status//node_state[@id='1']//transient_attributes//nvpair[@name='pgsql-status']
(/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3])
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
find_nvpair_attr_delegate: Match <nvpair id="status-1-pgsql-status"
name="pgsql-status" value="STOP"/>
Oct 12 18:41:29 [32391] pm1 crm_attribute: info: main: Read
pgsql-status=STOP
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32391-14-header
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32391-14-header
Oct 12 18:41:29 [32391] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32391-14-header
Oct 12 18:41:29 [32391] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:29 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32391-14)
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32391-14) state:2
Oct 12 18:41:29 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32391-14-header
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32391-14-header
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32391-14-header
pgsql(pgsql)[32155]: 2015/10/12_18:41:29 INFO: Changing pgsql-status
on pm1 : STOP->HS:alone.
Oct 12 18:41:29 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf235d0 for uid=0 gid=0 pid=32396
id=a721584f-639c-436f-bfb3-6550ac80f896
Oct 12 18:41:29 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32396-14)
Oct 12 18:41:29 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32396]
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:29 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:29 [32396] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:29 [32396] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:41:29 [31932] pm1 attrd: debug: crm_client_new:
Connecting 0xbcd700 for uid=0 gid=0 pid=32396
id=4af7f6f5-8650-4b8e-86c7-c4564f20f1d3
Oct 12 18:41:29 [31932] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (31932-32396-11)
Oct 12 18:41:29 [31932] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [32396]
Oct 12 18:41:29 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:29 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:29 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:29 [31932] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-status[pm1] = HS:alone (writer)
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-status=HS:alone for pm1
Oct 12 18:41:29 [32396] pm1 crm_attribute: info: main: Update
pgsql-status=HS:alone sent via attrd
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32396-14-header
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32396-14-header
Oct 12 18:41:29 [32396] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32396-14-header
Oct 12 18:41:29 [32396] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:29 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32396-14)
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32396-14) state:2
Oct 12 18:41:29 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32396-14-header
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32396-14-header
Oct 12 18:41:29 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32396-14-header
Oct 12 18:41:29 [31931] pm1 lrmd: debug: operation_finished:
pgsql_start_0:32155 - exited with rc=0
Oct 12 18:41:29 [31931] pm1 lrmd: notice: operation_finished:
pgsql_start_0:32155:stderr [ psql: FATAL: the database system is
starting up ]
Oct 12 18:41:29 [31931] pm1 lrmd: debug: operation_finished:
pgsql_start_0:32155:stdout [ -- empty -- ]
Oct 12 18:41:29 [31931] pm1 lrmd: info: log_finished: finished
- rsc:pgsql action:start call_id:15 pid:32155 exit-code:0
exec-time:1411ms queue-time:0ms
Oct 12 18:41:29 [31934] pm1 crmd: debug:
create_operation_update: do_update_resource: Updating resource pgsql
after start op complete (interval=0)
Oct 12 18:41:29 [31932] pm1 attrd: info: attrd_peer_update:
Setting pgsql-status[pm1]: STOP -> HS:alone from pm1
Oct 12 18:41:29 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[pgsql-status]=HS:alone (1 1 1 pm1)
Oct 12 18:41:29 [31932] pm1 attrd: debug: write_attribute:
Update: pm2[pgsql-status]=HS:alone (2 2 2 pm2)
Oct 12 18:41:29 [31932] pm1 attrd: info: write_attribute:
Sent update 12 with 2 changes for pgsql-status, id=<n/a>, set=(null)
Oct 12 18:41:29 [31932] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31932-32396-11)
Oct 12 18:41:29 [31932] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31932-32396-11) state:2
Oct 12 18:41:29 [31932] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:29 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-31932-32396-11-header
Oct 12 18:41:29 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-31932-32396-11-header
Oct 12 18:41:29 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-31932-32396-11-header
Oct 12 18:41:29 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/12)
Oct 12 18:41:29 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:29 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.21 2
Oct 12 18:41:29 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.22 (null)
Oct 12 18:41:29 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=22
Oct 12 18:41:29 [31929] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-pgsql-status']:
@value=HS:alone
Oct 12 18:41:29 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.22 to 0.19.21
Oct 12 18:41:29 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/12, version=0.19.22)
Oct 12 18:41:29 [31932] pm1 attrd: info: attrd_cib_callback:
Update 12 for pgsql-status: OK (0)
Oct 12 18:41:29 [31932] pm1 attrd: info: attrd_cib_callback:
Update 12 for pgsql-status[pm1]=HS:alone: OK (0)
Oct 12 18:41:29 [31932] pm1 attrd: info: attrd_cib_callback:
Update 12 for pgsql-status[pm2]=HS:alone: OK (0)
Oct 12 18:41:29 [32431] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:29 [32431] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-31913-32431-24-header
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-31913-32431-24-header
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-31913-32431-24-header
Oct 12 18:41:29 [32431] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-32431-24-header
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-32431-24-header
Oct 12 18:41:29 [32431] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-32431-24-header
Oct 12 18:41:29 [32431] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:29 [31934] pm1 crmd: info: action_synced_wait:
Managed pgsql_meta-data_0 process 32397 exited with rc=0
Oct 12 18:41:29 [31934] pm1 crmd: notice: process_lrm_event:
Operation pgsql_start_0: ok (node=pm1, call=15, rc=0, cib-update=41,
confirmed=true)
Oct 12 18:41:29 [31934] pm1 crmd: debug: process_lrm_event:
pm1-pgsql_start_0:15 [ psql: FATAL: the database system is starting up\n ]
Oct 12 18:41:29 [31934] pm1 crmd: debug:
update_history_cache: Updating history for 'pgsql' with start op
Oct 12 18:41:29 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.21 -> 0.19.22 (S_TRANSITION_ENGINE)
Oct 12 18:41:29 [31934] pm1 crmd: info:
abort_transition_graph: Transition aborted by status-1-pgsql-status,
pgsql-status=HS:alone: Transient attribute change (modify cib=0.19.22,
source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']/nvpair[@id='status-1-pgsql-status'],
0)
Oct 12 18:41:29 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=4, Pending=1, Fired=0, Skipped=0, Incomplete=7,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:29 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/41)
Oct 12 18:41:29 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:29 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.22 2
Oct 12 18:41:29 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.23 (null)
Oct 12 18:41:29 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=23
Oct 12 18:41:29 [31929] pm1 cib: info: cib_perform_op: +
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='pgsql']/lrm_rsc_op[@id='pgsql_last_0']:
@operation_key=pgsql_start_0, @operation=start,
@transition-key=5:4:0:b020e8a2-0497-404a-b66a-e629cc7d3213,
@transition-magic=0:0;5:4:0:b020e8a2-0497-404a-b66a-e629cc7d3213,
@call-id=15, @rc-code=0, @exec-time=1411
Oct 12 18:41:29 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/41, version=0.19.23)
Oct 12 18:41:29 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.23 to 0.19.22
Oct 12 18:41:29 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.22 -> 0.19.23 (S_TRANSITION_ENGINE)
Oct 12 18:41:29 [31934] pm1 crmd: info: match_graph_event:
Action pgsql_start_0 (5) confirmed on pm1 (rc=0)
Oct 12 18:41:29 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 10 fired and confirmed
Oct 12 18:41:29 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=5, Pending=0, Fired=1, Skipped=0, Incomplete=6,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:29 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 13 fired and confirmed
Oct 12 18:41:29 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=6, Pending=0, Fired=1, Skipped=0, Incomplete=5,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:29 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 44: notify pgsql_post_notify_start_0 on pm1 (local)
Oct 12 18:41:29 [31934] pm1 crmd: info: do_lrm_rsc_op:
Performing key=44:4:0:b020e8a2-0497-404a-b66a-e629cc7d3213 op=pgsql_notify_0
Oct 12 18:41:29 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=16, reply=1, notify=0, exit=4203648
Oct 12 18:41:29 [31931] pm1 lrmd: info: log_execute: executing
- rsc:pgsql action:notify call_id:16
Oct 12 18:41:29 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 45: notify pgsql:1_post_notify_start_0 on pm2
Oct 12 18:41:29 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=7, Pending=2, Fired=2, Skipped=0, Incomplete=3,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:29 [32467] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:29 [32467] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-31913-32467-24-header
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-31913-32467-24-header
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-31913-32467-24-header
Oct 12 18:41:29 [32467] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-32467-24-header
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-32467-24-header
Oct 12 18:41:29 [32467] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-32467-24-header
Oct 12 18:41:29 [32467] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
pgsql(pgsql)[32433]: 2015/10/12_18:41:29 DEBUG: notify: post for start
Oct 12 18:41:29 [31931] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:32433 - exited with rc=0
Oct 12 18:41:29 [31931] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:32433:stderr [ -- empty -- ]
Oct 12 18:41:29 [31931] pm1 lrmd: debug: operation_finished:
pgsql_notify_0:32433:stdout [ -- empty -- ]
Oct 12 18:41:29 [31931] pm1 lrmd: info: log_finished: finished
- rsc:pgsql action:notify call_id:16 pid:32433 exit-code:0
exec-time:82ms queue-time:0ms
Oct 12 18:41:29 [31934] pm1 crmd: debug:
create_operation_update: send_direct_ack: Updating resource pgsql
after notify op complete (interval=0)
Oct 12 18:41:29 [31934] pm1 crmd: debug: send_direct_ack:
ACK'ing resource op pgsql_notify_0 from
44:4:0:b020e8a2-0497-404a-b66a-e629cc7d3213: lrm_invoke-lrmd-1444664489-30
Oct 12 18:41:29 [31934] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664489-30 from pm1
Oct 12 18:41:29 [31934] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (44) confirmed on pm1 (rc=0)
Oct 12 18:41:29 [31934] pm1 crmd: notice: process_lrm_event:
Operation pgsql_notify_0: ok (node=pm1, call=16, rc=0, cib-update=0,
confirmed=true)
Oct 12 18:41:29 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=8, Pending=1, Fired=0, Skipped=0, Incomplete=3,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:29 [31934] pm1 crmd: debug: process_te_message:
Processing (N)ACK lrm_invoke-lrmd-1444664489-6 from pm2
Oct 12 18:41:29 [31934] pm1 crmd: info: match_graph_event:
Action pgsql_notify_0 (45) confirmed on pm2 (rc=0)
Oct 12 18:41:29 [31934] pm1 crmd: debug: te_pseudo_action:
Pseudo action 14 fired and confirmed
Oct 12 18:41:29 [31934] pm1 crmd: debug: run_graph: Transition
4 (Complete=9, Pending=0, Fired=1, Skipped=0, Incomplete=2,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): In-progress
Oct 12 18:41:29 [31934] pm1 crmd: notice: run_graph: Transition
4 (Complete=10, Pending=0, Fired=0, Skipped=2, Incomplete=2,
Source=/var/lib/pacemaker/pengine/pe-input-111.bz2): Stopped
Oct 12 18:41:29 [31934] pm1 crmd: debug: te_graph_trigger:
Transition 4 is now complete
Oct 12 18:41:29 [31934] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:41:29 [31934] pm1 crmd: debug: notify_crmd:
Transition 4 status: restart - Transient attribute change
Oct 12 18:41:29 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:41:29 [31934] pm1 crmd: info:
do_state_transition: State transition S_TRANSITION_ENGINE ->
S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:41:29 [31934] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:41:29 [31934] pm1 crmd: debug: do_pe_invoke: Query
42: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:41:29 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:29 [31934] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=42,
ref=pe_calc-dc-1444664489-31, seq=172, quorate=1
Oct 12 18:41:29 [31933] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:29 [31933] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:29 [31933] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:29 [31933] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:29 [31933] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:29 [31933] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:29 [31933] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:29 [31933] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:29 [31933] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:29 [31933] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:41:29 [31933] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:41:29 [31933] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:41:29 [31933] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:41:29 [31933] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:41:29 [31933] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:41:29 [31933] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:41:29 [31933] pm1 pengine: info: short_print:
Slaves: [ pm1 pm2 ]
Oct 12 18:41:29 [31933] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:41:29 [31933] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:29 [31933] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:29 [31933] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:1: preferring current
location (node=pm1, weight=1000000)
Oct 12 18:41:29 [31933] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:0: preferring current
location (node=pm2, weight=1000000)
Oct 12 18:41:29 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:1
Oct 12 18:41:29 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:0
Oct 12 18:41:29 [31933] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:41:29 [31933] pm1 pengine: debug: master_color: pgsql:0
master score: -1000000
Oct 12 18:41:29 [31933] pm1 pengine: debug: master_color: pgsql:1
master score: -1000000
Oct 12 18:41:29 [31933] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:41:29 [31933] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:41:29 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:41:29 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:41:29 [31933] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:41:29 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:41:29 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:41:29 [31933] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:41:29 [31933] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:41:29 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm2
Oct 12 18:41:29 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm1
Oct 12 18:41:29 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:0 on pm2
Oct 12 18:41:29 [31933] pm1 pengine: info: RecurringOp: Start
recurring monitor (4s) for pgsql:1 on pm1
Oct 12 18:41:29 [31933] pm1 pengine: info: LogActions: Leave
pgsql:0 (Slave pm2)
Oct 12 18:41:29 [31933] pm1 pengine: info: LogActions: Leave
pgsql:1 (Slave pm1)
Oct 12 18:41:29 [31933] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:41:29 [31933] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:41:29 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:0 (aka. pgsql)
Oct 12 18:41:29 [31933] pm1 pengine: debug: action2xml: Using
anonymous clone name pgsql for pgsql:1 (aka. pgsql)
Oct 12 18:41:29 [31933] pm1 pengine: notice: process_pe_message:
Calculated Transition 5: /var/lib/pacemaker/pengine/pe-input-112.bz2
Oct 12 18:41:29 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:41:29 [31934] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:41:29 [31934] pm1 crmd: debug: unpack_graph: Unpacked
transition 5: 2 actions in 2 synapses
Oct 12 18:41:29 [31934] pm1 crmd: info: do_te_invoke:
Processing graph 5 (ref=pe_calc-dc-1444664489-31) derived from
/var/lib/pacemaker/pengine/pe-input-112.bz2
Oct 12 18:41:29 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 7: monitor pgsql_monitor_4000 on pm2
Oct 12 18:41:29 [31934] pm1 crmd: notice: te_rsc_command:
Initiating action 10: monitor pgsql_monitor_4000 on pm1 (local)
Oct 12 18:41:29 [31934] pm1 crmd: info: do_lrm_rsc_op:
Performing key=10:5:0:b020e8a2-0497-404a-b66a-e629cc7d3213
op=pgsql_monitor_4000
Oct 12 18:41:29 [31931] pm1 lrmd: debug:
process_lrmd_message: Processed lrmd_rsc_exec operation from
767e1a8c-765b-42f5-92f3-5db5fb068a58: rc=17, reply=1, notify=0, exit=4203648
Oct 12 18:41:29 [31931] pm1 lrmd: debug: log_execute: executing
- rsc:pgsql action:monitor call_id:17
Oct 12 18:41:29 [31934] pm1 crmd: debug: run_graph: Transition
5 (Complete=0, Pending=2, Fired=2, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-112.bz2): In-progress
Oct 12 18:41:29 [32560] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:29 [32560] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-31913-32560-24-header
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-31913-32560-24-header
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-31913-32560-24-header
Oct 12 18:41:29 [32560] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-32560-24-header
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-32560-24-header
Oct 12 18:41:29 [32560] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-32560-24-header
Oct 12 18:41:29 [32560] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
pgsql(pgsql)[32526]: 2015/10/12_18:41:30 DEBUG: PostgreSQL is running
as a hot standby.
Oct 12 18:41:30 [32634] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf22f80 for uid=0 gid=0 pid=32634
id=4a4bce90-7ebf-48ed-bd04-4a7c675fb565
Oct 12 18:41:30 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32634-14)
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32634]
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32634] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32634] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32634] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32634] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [32634] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:30 [32634] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:30 [32634] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:30 [32634] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:30 [32634] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:30 [32634] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:30 [32634] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:30 [32634] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:30 [32634] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:30 [32634] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:41:30 [32634] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:41:30 [32634] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32634-14)
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32634-14) state:2
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-31929-32634-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-31929-32634-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-31929-32634-14-header
pgsql(pgsql)[32526]: 2015/10/12_18:41:30 INFO: Master does not exist.
Oct 12 18:41:30 [32645] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf22f80 for uid=0 gid=0 pid=32645
id=d33727e5-c360-46be-82cb-7642cc780526
Oct 12 18:41:30 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32645-14)
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32645]
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32645] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32645] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32645] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32645] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [32645] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:30 [32645] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:30 [32645] pm1 crm_mon: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:30 [32645] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:30 [32645] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:30 [32645] pm1 crm_mon: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:30 [32645] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:30 [32645] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:30 [32645] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:30 [32645] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:41:30 [32645] pm1 crm_mon: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:41:30 [32645] pm1 crm_mon: debug: get_xpath_object:
No match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32645-14)
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32645-14) state:2
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-31929-32645-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-31929-32645-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-31929-32645-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf22f80 for uid=0 gid=0 pid=32647
id=76e45818-7cd4-45be-aa9f-cb9cd705bf57
Oct 12 18:41:30 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32647-14)
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32647]
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:30 [32647] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/status//node_state[@id='1']//transient_attributes//nvpair[@name='pgsql-status']
(/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3])
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
find_nvpair_attr_delegate: Match <nvpair id="status-1-pgsql-status"
name="pgsql-status" value="HS:alone"/>
Oct 12 18:41:30 [32647] pm1 crm_attribute: info: main: Read
pgsql-status=HS:alone
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32647-14-header
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32647-14-header
Oct 12 18:41:30 [32647] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32647-14-header
Oct 12 18:41:30 [32647] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32647-14)
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32647-14) state:2
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32647-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32647-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32647-14-header
pgsql(pgsql)[32526]: 2015/10/12_18:41:30 DEBUG: Checking if I have a
master right.
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf22f80 for uid=0 gid=0 pid=32653
id=03ad8b54-9fa0-400f-8b70-cff84b5240d5
Oct 12 18:41:30 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32653-14)
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32653]
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:30 [32653] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/configuration/nodes//node[@id='1']//instance_attributes//nvpair[@name='pgsql-data-status']
(/cib/configuration/nodes/node[1]/instance_attributes/nvpair)
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
find_nvpair_attr_delegate: Match <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:30 [32653] pm1 crm_attribute: info: main: Read
pgsql-data-status=LATEST
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32653-14-header
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32653-14-header
Oct 12 18:41:30 [32653] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32653-14-header
Oct 12 18:41:30 [32653] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32653-14)
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32653-14) state:2
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32653-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32653-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32653-14-header
pgsql(pgsql)[32526]: 2015/10/12_18:41:30 INFO: My data status=LATEST.
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf22f80 for uid=0 gid=0 pid=32699
id=246b8e8a-1490-4b90-88fe-dfbd17366126
Oct 12 18:41:30 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32699-14)
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32699]
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:30 [32699] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:30 [32699] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:41:30 [31932] pm1 attrd: debug: crm_client_new:
Connecting 0xbcd700 for uid=0 gid=0 pid=32699
id=728ec38d-874a-404a-a373-7b10c7eb96fd
Oct 12 18:41:30 [31932] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (31932-32699-11)
Oct 12 18:41:30 [31932] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [32699]
Oct 12 18:41:30 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:30 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:30 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:30 [31932] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-xlog-loc[pm1] =
0000000000000000 (writer)
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-xlog-loc=0000000000000000
for pm1
Oct 12 18:41:30 [32699] pm1 crm_attribute: info: main: Update
pgsql-xlog-loc=0000000000000000 sent via attrd
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:30 [31932] pm1 attrd: info: attrd_peer_update:
Setting pgsql-xlog-loc[pm1]: (null) -> 0000000000000000 from pm1
Oct 12 18:41:30 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[pgsql-xlog-loc]=0000000000000000 (1 1 1 pm1)
Oct 12 18:41:30 [31932] pm1 attrd: debug: write_attribute:
Update: pm2[pgsql-xlog-loc]=(null) (2 2 2 pm2)
Oct 12 18:41:30 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/13)
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_process_modify:
Destroying
/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[5]
Oct 12 18:41:30 [31932] pm1 attrd: info: write_attribute:
Sent update 13 with 2 changes for pgsql-xlog-loc, id=<n/a>, set=(null)
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.23 2
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.24 (null)
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=24
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1']:
<nvpair id="status-1-pgsql-xlog-loc" name="pgsql-xlog-loc"
value="0000000000000000"/>
Oct 12 18:41:30 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/13, version=0.19.24)
Oct 12 18:41:30 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.24 to 0.19.23
Oct 12 18:41:30 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.23 -> 0.19.24 (S_TRANSITION_ENGINE)
Oct 12 18:41:30 [31934] pm1 crmd: debug:
update_abort_priority: Abort priority upgraded from 0 to 1000000
Oct 12 18:41:30 [31934] pm1 crmd: debug:
update_abort_priority: Abort action done superseded by restart:
Transient attribute change
Oct 12 18:41:30 [31934] pm1 crmd: notice:
abort_transition_graph: Transition aborted by
status-1-pgsql-xlog-loc, pgsql-xlog-loc=0000000000000000: Transient
attribute change (create cib=0.19.24, source=te_update_diff:391,
path=/cib/status/node_state[@id='1']/transient_attributes[@id='1']/instance_attributes[@id='status-1'],
0)
Oct 12 18:41:30 [31934] pm1 crmd: debug: run_graph: Transition
5 (Complete=0, Pending=2, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-112.bz2): In-progress
Oct 12 18:41:30 [31932] pm1 attrd: info: attrd_cib_callback:
Update 13 for pgsql-xlog-loc: OK (0)
Oct 12 18:41:30 [31932] pm1 attrd: info: attrd_cib_callback:
Update 13 for pgsql-xlog-loc[pm1]=0000000000000000: OK (0)
Oct 12 18:41:30 [31932] pm1 attrd: info: attrd_cib_callback:
Update 13 for pgsql-xlog-loc[pm2]=(null): OK (0)
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32699-14-header
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32699-14-header
Oct 12 18:41:30 [32699] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32699-14-header
Oct 12 18:41:30 [32699] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:30 [31932] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31932-32699-11)
Oct 12 18:41:30 [31932] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31932-32699-11) state:2
Oct 12 18:41:30 [31932] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:30 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-31932-32699-11-header
Oct 12 18:41:30 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-31932-32699-11-header
Oct 12 18:41:30 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-31932-32699-11-header
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32699-14)
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32699-14) state:2
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32699-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32699-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32699-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf22f80 for uid=0 gid=0 pid=32703
id=606130e1-8243-4d58-880c-c957bd59d2b5
Oct 12 18:41:30 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32703-14)
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32703]
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug: query_node_uuid:
pg1 is not a cluster node, checking to see if remote-node
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //nvpair[@name='remote-node'][@value='pg1'] does not exist
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //primitive[@type='remote'][@provider='pacemaker'][@id='pg1']
does not exist
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //status//node_state[@id='pg1'][@remote_node='true'] does not
exist
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug: query_node_uuid:
pg1 is not a remote node either
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug: query_node_uuid:
Could not map name=pg1 to a UUID: Operation not permitted
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32703-14-header
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32703-14-header
Oct 12 18:41:30 [32703] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32703-14-header
Oct 12 18:41:30 [32703] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32703-14)
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32703-14) state:2
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32703-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32703-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32703-14-header
pgsql(pgsql)[32526]: 2015/10/12_18:41:30 WARNING: Can't get pg1 xlog
location.
Oct 12 18:41:30 [31932] pm1 attrd: info: attrd_peer_update:
Setting pgsql-xlog-loc[pm2]: (null) -> 0000000000000000 from pm2
Oct 12 18:41:30 [31932] pm1 attrd: debug: write_attribute:
Update: pm1[pgsql-xlog-loc]=0000000000000000 (1 1 1 pm1)
Oct 12 18:41:30 [31932] pm1 attrd: debug: write_attribute:
Update: pm2[pgsql-xlog-loc]=0000000000000000 (2 2 2 pm2)
Oct 12 18:41:30 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/attrd/14)
Oct 12 18:41:30 [31932] pm1 attrd: info: write_attribute:
Sent update 14 with 2 changes for pgsql-xlog-loc, id=<n/a>, set=(null)
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.24 2
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.25 (null)
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=25
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2']:
<nvpair id="status-2-pgsql-xlog-loc" name="pgsql-xlog-loc"
value="0000000000000000"/>
Oct 12 18:41:30 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.25 to 0.19.24
Oct 12 18:41:30 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.24 -> 0.19.25 (S_TRANSITION_ENGINE)
Oct 12 18:41:30 [31934] pm1 crmd: info:
abort_transition_graph: Transition aborted by
status-2-pgsql-xlog-loc, pgsql-xlog-loc=0000000000000000: Transient
attribute change (create cib=0.19.25, source=te_update_diff:391,
path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'],
0)
Oct 12 18:41:30 [31934] pm1 crmd: debug: run_graph: Transition
5 (Complete=0, Pending=2, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-112.bz2): In-progress
Oct 12 18:41:30 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/attrd/14, version=0.19.25)
Oct 12 18:41:30 [31932] pm1 attrd: info: attrd_cib_callback:
Update 14 for pgsql-xlog-loc: OK (0)
Oct 12 18:41:30 [31932] pm1 attrd: info: attrd_cib_callback:
Update 14 for pgsql-xlog-loc[pm1]=0000000000000000: OK (0)
Oct 12 18:41:30 [31932] pm1 attrd: info: attrd_cib_callback:
Update 14 for pgsql-xlog-loc[pm2]=0000000000000000: OK (0)
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xf22f80 for uid=0 gid=0 pid=32709
id=7dca1174-66a6-4c74-bcf8-574e60e5ab5e
Oct 12 18:41:30 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-32709-14)
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [32709]
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug: qb_rb_open_2:
shm size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug: query_node_uuid:
pg2 is not a cluster node, checking to see if remote-node
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //nvpair[@name='remote-node'][@value='pg2'] does not exist
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //primitive[@type='remote'][@provider='pacemaker'][@id='pg2']
does not exist
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //status//node_state[@id='pg2'][@remote_node='true'] does not
exist
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug: query_node_uuid:
pg2 is not a remote node either
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug: query_node_uuid:
Could not map name=pg2 to a UUID: Operation not permitted
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-request-31929-32709-14-header
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-response-31929-32709-14-header
Oct 12 18:41:30 [32709] pm1 crm_attribute: debug: qb_rb_close:
Closing ringbuffer: /dev/shm/qb-cib_rw-event-31929-32709-14-header
Oct 12 18:41:30 [32709] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:30 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-32709-14)
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-32709-14) state:2
Oct 12 18:41:30 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-32709-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-32709-14-header
Oct 12 18:41:30 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-32709-14-header
pgsql(pgsql)[32526]: 2015/10/12_18:41:30 WARNING: Can't get pg2 xlog
location.
Oct 12 18:41:30 [31931] pm1 lrmd: debug: operation_finished:
pgsql_monitor_4000:32526 - exited with rc=0
Oct 12 18:41:30 [31931] pm1 lrmd: debug: operation_finished:
pgsql_monitor_4000:32526:stderr [ -- empty -- ]
Oct 12 18:41:30 [31931] pm1 lrmd: debug: operation_finished:
pgsql_monitor_4000:32526:stdout [ -- empty -- ]
Oct 12 18:41:30 [31931] pm1 lrmd: debug: log_finished: finished
- rsc:pgsql action:monitor call_id:17 pid:32526 exit-code:0
exec-time:228ms queue-time:0ms
Oct 12 18:41:30 [31934] pm1 crmd: debug:
create_operation_update: do_update_resource: Updating resource pgsql
after monitor op complete (interval=4000)
Oct 12 18:41:30 [31934] pm1 crmd: info: process_lrm_event:
Operation pgsql_monitor_4000: ok (node=pm1, call=17, rc=0,
cib-update=43, confirmed=false)
Oct 12 18:41:30 [31934] pm1 crmd: debug:
update_history_cache: Updating history for 'pgsql' with monitor op
Oct 12 18:41:30 [31929] pm1 cib: info:
cib_process_request: Forwarding cib_modify operation for section
status to master (origin=local/crmd/43)
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.25 2
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.26 (null)
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=26
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='pgsql']:
<lrm_rsc_op id="pgsql_monitor_4000" operation_key="pgsql_monitor_4000"
operation="monitor" crm-debug-origin="do_update_resource"
crm_feature_set="3.0.10"
transition-key="10:5:0:b020e8a2-0497-404a-b66a-e629cc7d3213"
transition-magic="0:0;10:5:0:b020e8a2-0497-404a-b66a-e629cc7d3213"
on_node="pm1" call-id="17" rc-code="0" op-status="0" interval="4000" last
Oct 12 18:41:30 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.26 to 0.19.25
Oct 12 18:41:30 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm1/crmd/43, version=0.19.26)
Oct 12 18:41:30 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.25 -> 0.19.26 (S_TRANSITION_ENGINE)
Oct 12 18:41:30 [31934] pm1 crmd: info: match_graph_event:
Action pgsql_monitor_4000 (10) confirmed on pm1 (rc=0)
Oct 12 18:41:30 [31934] pm1 crmd: debug: run_graph: Transition
5 (Complete=1, Pending=1, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-112.bz2): In-progress
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: Diff:
--- 0.19.26 2
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: Diff:
+++ 0.19.27 (null)
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: +
/cib: @num_updates=27
Oct 12 18:41:30 [31929] pm1 cib: info: cib_perform_op: ++
/cib/status/node_state[@id='2']/lrm[@id='2']/lrm_resources/lrm_resource[@id='pgsql']:
<lrm_rsc_op id="pgsql_monitor_4000" operation_key="pgsql_monitor_4000"
operation="monitor" crm-debug-origin="do_update_resource"
crm_feature_set="3.0.10"
transition-key="7:5:0:b020e8a2-0497-404a-b66a-e629cc7d3213"
transition-magic="0:0;7:5:0:b020e8a2-0497-404a-b66a-e629cc7d3213"
on_node="pm2" call-id="17" rc-code="0" op-status="0" interval="4000" last-r
Oct 12 18:41:30 [31930] pm1 stonith-ng: debug:
xml_patch_version_check: Can apply patch 0.19.27 to 0.19.26
Oct 12 18:41:30 [31934] pm1 crmd: debug: te_update_diff:
Processing (cib_modify) diff: 0.19.26 -> 0.19.27 (S_TRANSITION_ENGINE)
Oct 12 18:41:30 [31934] pm1 crmd: info: match_graph_event:
Action pgsql_monitor_4000 (7) confirmed on pm2 (rc=0)
Oct 12 18:41:30 [31934] pm1 crmd: notice: run_graph: Transition
5 (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-112.bz2): Complete
Oct 12 18:41:30 [31934] pm1 crmd: debug: te_graph_trigger:
Transition 5 is now complete
Oct 12 18:41:30 [31934] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:41:30 [31934] pm1 crmd: debug: notify_crmd:
Transition 5 status: restart - Transient attribute change
Oct 12 18:41:30 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:41:30 [31934] pm1 crmd: info:
do_state_transition: State transition S_TRANSITION_ENGINE ->
S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:41:30 [31934] pm1 crmd: debug:
do_state_transition: All 2 cluster nodes are eligible to run resources.
Oct 12 18:41:30 [31934] pm1 crmd: debug: do_pe_invoke: Query
44: Requesting the current CIB: S_POLICY_ENGINE
Oct 12 18:41:30 [31929] pm1 cib: info:
cib_process_request: Completed cib_modify operation for section
status: OK (rc=0, origin=pm2/crmd/15, version=0.19.27)
Oct 12 18:41:30 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:30 [31934] pm1 crmd: debug:
do_pe_invoke_callback: Invoking the PE: query=44,
ref=pe_calc-dc-1444664490-34, seq=172, quorate=1
Oct 12 18:41:30 [31933] pm1 pengine: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:30 [31933] pm1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:30 [31933] pm1 pengine: debug: unpack_config: Stop
all active resources: false
Oct 12 18:41:30 [31933] pm1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:30 [31933] pm1 pengine: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:30 [31933] pm1 pengine: notice: unpack_config: On loss
of CCM Quorum: Ignore
Oct 12 18:41:30 [31933] pm1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:30 [31933] pm1 pengine: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:30 [31933] pm1 pengine: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:30 [31933] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:41:30 [31933] pm1 pengine: debug:
find_anonymous_clone: Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:41:30 [31933] pm1 pengine: info: clone_print:
Master/Slave Set: msPostgresql [pgsql]
Oct 12 18:41:30 [31933] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:41:30 [31933] pm1 pengine: debug: native_active:
Resource pgsql:0 active on pm2
Oct 12 18:41:30 [31933] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:41:30 [31933] pm1 pengine: debug: native_active:
Resource pgsql:1 active on pm1
Oct 12 18:41:30 [31933] pm1 pengine: info: short_print:
Slaves: [ pm1 pm2 ]
Oct 12 18:41:30 [31933] pm1 pengine: info: group_print: Resource
Group: master-group
Oct 12 18:41:30 [31933] pm1 pengine: info: native_print:
vip-master (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:30 [31933] pm1 pengine: info: native_print:
vip-rep (ocf::heartbeat:IPaddr2): Stopped
Oct 12 18:41:30 [31933] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:1: preferring current
location (node=pm1, weight=1000000)
Oct 12 18:41:30 [31933] pm1 pengine: debug:
common_apply_stickiness: Resource pgsql:0: preferring current
location (node=pm2, weight=1000000)
Oct 12 18:41:30 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm1 to pgsql:1
Oct 12 18:41:30 [31933] pm1 pengine: debug: native_assign_node:
Assigning pm2 to pgsql:0
Oct 12 18:41:30 [31933] pm1 pengine: debug: clone_color: Allocated
2 msPostgresql instances of a possible 2
Oct 12 18:41:30 [31933] pm1 pengine: debug: master_color: pgsql:0
master score: -1000000
Oct 12 18:41:30 [31933] pm1 pengine: debug: master_color: pgsql:1
master score: -1000000
Oct 12 18:41:30 [31933] pm1 pengine: info: master_color:
msPostgresql: Promoted 0 instances of a possible 1 to master
Oct 12 18:41:30 [31933] pm1 pengine: info: rsc_merge_weights:
vip-master: Rolling back scores from vip-rep
Oct 12 18:41:30 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-master are unavailable, unclean or shutting
down (pm1: 1, -1000000)
Oct 12 18:41:30 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-master
Oct 12 18:41:30 [31933] pm1 pengine: info: native_color: Resource
vip-master cannot run anywhere
Oct 12 18:41:30 [31933] pm1 pengine: debug: native_assign_node:
All nodes for resource vip-rep are unavailable, unclean or shutting down
(pm1: 1, -1000000)
Oct 12 18:41:30 [31933] pm1 pengine: debug: native_assign_node:
Could not allocate a node for vip-rep
Oct 12 18:41:30 [31933] pm1 pengine: info: native_color: Resource
vip-rep cannot run anywhere
Oct 12 18:41:30 [31933] pm1 pengine: debug:
master_create_actions: Creating actions for msPostgresql
Oct 12 18:41:30 [31933] pm1 pengine: info: LogActions: Leave
pgsql:0 (Slave pm2)
Oct 12 18:41:30 [31933] pm1 pengine: info: LogActions: Leave
pgsql:1 (Slave pm1)
Oct 12 18:41:30 [31933] pm1 pengine: info: LogActions: Leave
vip-master (Stopped)
Oct 12 18:41:30 [31933] pm1 pengine: info: LogActions: Leave
vip-rep (Stopped)
Oct 12 18:41:30 [31933] pm1 pengine: notice: process_pe_message:
Calculated Transition 6: /var/lib/pacemaker/pengine/pe-input-113.bz2
Oct 12 18:41:30 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:41:30 [31934] pm1 crmd: info:
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Oct 12 18:41:30 [31934] pm1 crmd: debug: unpack_graph: Unpacked
transition 6: 0 actions in 0 synapses
Oct 12 18:41:30 [31934] pm1 crmd: info: do_te_invoke:
Processing graph 6 (ref=pe_calc-dc-1444664490-34) derived from
/var/lib/pacemaker/pengine/pe-input-113.bz2
Oct 12 18:41:30 [31934] pm1 crmd: debug: print_graph: Empty
transition graph
Oct 12 18:41:30 [31934] pm1 crmd: notice: run_graph: Transition
6 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-113.bz2): Complete
Oct 12 18:41:30 [31934] pm1 crmd: debug: te_graph_trigger:
Transition 6 is now complete
Oct 12 18:41:30 [31934] pm1 crmd: debug: notify_crmd:
Processing transition completion in state S_TRANSITION_ENGINE
Oct 12 18:41:30 [31934] pm1 crmd: debug: notify_crmd:
Transition 6 status: done - <null>
Oct 12 18:41:30 [31934] pm1 crmd: debug: s_crmd_fsa: Processing
I_TE_SUCCESS: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Oct 12 18:41:30 [31934] pm1 crmd: info: do_log: FSA: Input
I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE
Oct 12 18:41:30 [31934] pm1 crmd: notice:
do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [
input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 12 18:41:30 [31934] pm1 crmd: debug:
do_state_transition: Starting PEngine Recheck Timer
Oct 12 18:41:30 [31934] pm1 crmd: debug: crm_timer_start:
Started PEngine Recheck Timer (I_PE_CALC:900000ms), src=68
Oct 12 18:41:34 [31931] pm1 lrmd: debug:
recurring_action_timer: Scheduling another invocation of
pgsql_monitor_4000
Oct 12 18:41:34 [32755] pm1 crm_node: info: get_cluster_type:
Verifying cluster type: 'corosync'
Oct 12 18:41:34 [32755] pm1 crm_node: info: get_cluster_type:
Assuming an active 'corosync' cluster
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-request-31913-32755-24-header
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-response-31913-32755-24-header
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cpg-event-31913-32755-24-header
Oct 12 18:41:34 [32755] pm1 crm_node: debug: get_local_nodeid:
Local nodeid is 1
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_open_2: shm
size:1048589; real_size:1052672; rb->word_size:263168
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_ipcc_disconnect:
qb_ipcc_disconnect()
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-request-31913-32755-24-header
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-response-31913-32755-24-header
Oct 12 18:41:34 [32755] pm1 crm_node: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cmap-event-31913-32755-24-header
Oct 12 18:41:34 [32755] pm1 crm_node: info: crm_xml_cleanup:
Cleaning up memory from libxml2
pgsql(pgsql)[32721]: 2015/10/12_18:41:34 DEBUG: PostgreSQL is running
as a hot standby.
Oct 12 18:41:34 [366] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xc60ca0 for uid=0 gid=0 pid=366
id=58c2abc1-44cd-4605-b9c1-58e30812025c
Oct 12 18:41:34 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-366-14)
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [366]
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [366] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [366] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [366] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [366] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [366] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:34 [366] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:34 [366] pm1 crm_mon: debug: unpack_config: Stop all
active resources: false
Oct 12 18:41:34 [366] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:34 [366] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:34 [366] pm1 crm_mon: notice: unpack_config: On
loss of CCM Quorum: Ignore
Oct 12 18:41:34 [366] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:34 [366] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:34 [366] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:34 [366] pm1 crm_mon: debug: find_anonymous_clone:
Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:41:34 [366] pm1 crm_mon: debug: find_anonymous_clone:
Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:41:34 [366] pm1 crm_mon: debug: get_xpath_object: No
match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-366-14)
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-366-14) state:2
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-31929-366-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-31929-366-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-31929-366-14-header
pgsql(pgsql)[32721]: 2015/10/12_18:41:34 INFO: Master does not exist.
Oct 12 18:41:34 [380] pm1 crm_mon: info: main: Starting crm_mon
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xc60ca0 for uid=0 gid=0 pid=380
id=c3007aec-5d4c-46f1-9357-82b394049c2f
Oct 12 18:41:34 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-380-14)
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [380]
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [380] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [380] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [380] pm1 crm_mon: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [380] pm1 crm_mon: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [380] pm1 crm_mon: debug: unpack_config: STONITH
timeout: 60000
Oct 12 18:41:34 [380] pm1 crm_mon: debug: unpack_config: STONITH
of failed nodes is disabled
Oct 12 18:41:34 [380] pm1 crm_mon: debug: unpack_config: Stop all
active resources: false
Oct 12 18:41:34 [380] pm1 crm_mon: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Oct 12 18:41:34 [380] pm1 crm_mon: debug: unpack_config: Default
stickiness: 0
Oct 12 18:41:34 [380] pm1 crm_mon: notice: unpack_config: On
loss of CCM Quorum: Ignore
Oct 12 18:41:34 [380] pm1 crm_mon: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Oct 12 18:41:34 [380] pm1 crm_mon: info:
determine_online_status: Node pm2 is online
Oct 12 18:41:34 [380] pm1 crm_mon: info:
determine_online_status: Node pm1 is online
Oct 12 18:41:34 [380] pm1 crm_mon: debug: find_anonymous_clone:
Internally renamed pgsql on pm2 to pgsql:0
Oct 12 18:41:34 [380] pm1 crm_mon: debug: find_anonymous_clone:
Internally renamed pgsql on pm1 to pgsql:1
Oct 12 18:41:34 [380] pm1 crm_mon: debug: get_xpath_object: No
match for //nvpair[@name='expected-quorum-votes'] in /cib
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-380-14)
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-380-14) state:2
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-response-31929-380-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-event-31929-380-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_ro-request-31929-380-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xc60ca0 for uid=0 gid=0 pid=382
id=66d77c35-0d04-4ba6-9983-856c3c71fed4
Oct 12 18:41:34 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-382-14)
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [382]
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [382] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [382] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [382] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:34 [382] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/status//node_state[@id='1']//transient_attributes//nvpair[@name='pgsql-status']
(/cib/status/node_state[2]/transient_attributes/instance_attributes/nvpair[3])
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
find_nvpair_attr_delegate: Match <nvpair id="status-1-pgsql-status"
name="pgsql-status" value="HS:alone"/>
Oct 12 18:41:34 [382] pm1 crm_attribute: info: main: Read
pgsql-status=HS:alone
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:34 [382] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:34 [382] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-382-14-header
Oct 12 18:41:34 [382] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-382-14-header
Oct 12 18:41:34 [382] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-382-14-header
Oct 12 18:41:34 [382] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-382-14)
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-382-14) state:2
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-382-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-382-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-382-14-header
pgsql(pgsql)[32721]: 2015/10/12_18:41:34 DEBUG: Checking if I have a
master right.
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xc60ca0 for uid=0 gid=0 pid=388
id=312ffd54-4336-4608-be03-22277b5c7112
Oct 12 18:41:34 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-388-14)
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [388]
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [388] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [388] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [388] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:34 [388] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_process_xpath:
Processing cib_query op for
//cib/configuration/nodes//node[@id='1']//instance_attributes//nvpair[@name='pgsql-data-status']
(/cib/configuration/nodes/node[1]/instance_attributes/nvpair)
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
find_nvpair_attr_delegate: Match <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:34 [388] pm1 crm_attribute: info: main: Read
pgsql-data-status=LATEST
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:34 [388] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:34 [388] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-388-14-header
Oct 12 18:41:34 [388] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-388-14-header
Oct 12 18:41:34 [388] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-388-14-header
Oct 12 18:41:34 [388] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-388-14)
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-388-14) state:2
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-388-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-388-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-388-14-header
pgsql(pgsql)[32721]: 2015/10/12_18:41:34 INFO: My data status=LATEST.
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xc60ca0 for uid=0 gid=0 pid=434
id=c69889cc-d67c-4891-903b-ec9fdc54aad2
Oct 12 18:41:34 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-434-14)
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [434]
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [434] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [434] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [434] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:34 [434] pm1 crm_attribute: info: query_node_uuid:
Mapped pm1 to 1
Oct 12 18:41:34 [434] pm1 crm_attribute: info:
attrd_update_delegate: Connecting to cluster... 5 retries remaining
Oct 12 18:41:34 [31932] pm1 attrd: debug: crm_client_new:
Connecting 0xbcd700 for uid=0 gid=0 pid=434
id=e178792f-1f0d-4112-a570-a3830a3e3ccc
Oct 12 18:41:34 [31932] pm1 attrd: debug:
handle_new_connection: IPC credentials authenticated (31932-434-11)
Oct 12 18:41:34 [31932] pm1 attrd: debug:
qb_ipcs_shm_connect: connecting to client [434]
Oct 12 18:41:34 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:34 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:34 [31932] pm1 attrd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:34 [434] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:34 [434] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:34 [434] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Oct 12 18:41:34 [31932] pm1 attrd: debug:
attrd_client_update: Broadcasting pgsql-xlog-loc[pm1] =
0000000000000000 (writer)
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
attrd_update_delegate: Sent update: pgsql-xlog-loc=0000000000000000
for pm1
Oct 12 18:41:34 [434] pm1 crm_attribute: info: main: Update
pgsql-xlog-loc=0000000000000000 sent via attrd
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:34 [434] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:34 [434] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-434-14-header
Oct 12 18:41:34 [434] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-434-14-header
Oct 12 18:41:34 [434] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-434-14-header
Oct 12 18:41:34 [434] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-434-14)
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-434-14) state:2
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-434-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-434-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-434-14-header
Oct 12 18:41:34 [31932] pm1 attrd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31932-434-11)
Oct 12 18:41:34 [31932] pm1 attrd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31932-434-11) state:2
Oct 12 18:41:34 [31932] pm1 attrd: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:34 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-response-31932-434-11-header
Oct 12 18:41:34 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-event-31932-434-11-header
Oct 12 18:41:34 [31932] pm1 attrd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-attrd-request-31932-434-11-header
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xc60ca0 for uid=0 gid=0 pid=438
id=4b7a5a5b-4ead-4c0d-aed5-ace4f19499c5
Oct 12 18:41:34 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-438-14)
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [438]
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [438] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [438] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [438] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [438] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [438] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:34 [438] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:34 [438] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:34 [438] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:34 [438] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:34 [438] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:34 [438] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:34 [438] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:34 [438] pm1 crm_attribute: debug: query_node_uuid:
pg1 is not a cluster node, checking to see if remote-node
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //nvpair[@name='remote-node'][@value='pg1'] does not exist
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //primitive[@type='remote'][@provider='pacemaker'][@id='pg1']
does not exist
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //status//node_state[@id='pg1'][@remote_node='true'] does not
exist
Oct 12 18:41:34 [438] pm1 crm_attribute: debug: query_node_uuid:
pg1 is not a remote node either
Oct 12 18:41:34 [438] pm1 crm_attribute: debug: query_node_uuid:
Could not map name=pg1 to a UUID: Operation not permitted
Oct 12 18:41:34 [438] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:34 [438] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:34 [438] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-438-14-header
Oct 12 18:41:34 [438] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-438-14-header
Oct 12 18:41:34 [438] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-438-14-header
Oct 12 18:41:34 [438] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-438-14)
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-438-14) state:2
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-438-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-438-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-438-14-header
pgsql(pgsql)[32721]: 2015/10/12_18:41:34 WARNING: Can't get pg1 xlog
location.
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_new:
Connecting 0xc60ca0 for uid=0 gid=0 pid=445
id=cb8b4f4f-7461-41f0-a8b3-93848e28fcbe
Oct 12 18:41:34 [31929] pm1 cib: debug:
handle_new_connection: IPC credentials authenticated (31929-445-14)
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_shm_connect: connecting to client [445]
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [445] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [445] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [445] pm1 crm_attribute: debug: qb_rb_open_2: shm
size:524301; real_size:528384; rb->word_size:132096
Oct 12 18:41:34 [445] pm1 crm_attribute: debug:
cib_native_signon_raw: Connection to CIB successful
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [445] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nodes>
Oct 12 18:41:34 [445] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="1" uname="pm1">
Oct 12 18:41:34 [445] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <instance_attributes id="nodes-1">
Oct 12 18:41:34 [445] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <nvpair
id="nodes-1-pgsql-data-status" name="pgsql-data-status" value="LATEST"/>
Oct 12 18:41:34 [445] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </instance_attributes>
Oct 12 18:41:34 [445] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </node>
Oct 12 18:41:34 [445] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section <node id="2" uname="pm2"/>
Oct 12 18:41:34 [445] pm1 crm_attribute: debug:
get_cluster_node_uuid: Result section </nodes>
Oct 12 18:41:34 [445] pm1 crm_attribute: debug: query_node_uuid:
pg2 is not a cluster node, checking to see if remote-node
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //nvpair[@name='remote-node'][@value='pg2'] does not exist
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //primitive[@type='remote'][@provider='pacemaker'][@id='pg2']
does not exist
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:34 [31929] pm1 cib: debug: cib_process_xpath:
cib_query: //status//node_state[@id='pg2'][@remote_node='true'] does not
exist
Oct 12 18:41:34 [445] pm1 crm_attribute: debug: query_node_uuid:
pg2 is not a remote node either
Oct 12 18:41:34 [445] pm1 crm_attribute: debug: query_node_uuid:
Could not map name=pg2 to a UUID: Operation not permitted
Oct 12 18:41:34 [445] pm1 crm_attribute: debug:
cib_native_signoff: Signing out of the CIB Service
Oct 12 18:41:34 [445] pm1 crm_attribute: debug:
qb_ipcc_disconnect: qb_ipcc_disconnect()
Oct 12 18:41:34 [445] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-445-14-header
Oct 12 18:41:34 [445] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-445-14-header
Oct 12 18:41:34 [445] pm1 crm_attribute: debug: qb_rb_close: Closing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-445-14-header
Oct 12 18:41:34 [445] pm1 crm_attribute: info: crm_xml_cleanup:
Cleaning up memory from libxml2
Oct 12 18:41:34 [31929] pm1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (31929-445-14)
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(31929-445-14) state:2
Oct 12 18:41:34 [31929] pm1 cib: debug: crm_client_destroy:
Destroying 0 events
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-31929-445-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-31929-445-14-header
Oct 12 18:41:34 [31929] pm1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-31929-445-14-header
pgsql(pgsql)[32721]: 2015/10/12_18:41:34 WARNING: Can't get pg2 xlog
location.
Oct 12 18:41:34 [31931] pm1 lrmd: debug: operation_finished:
pgsql_monitor_4000:32721 - exited with rc=0
Oct 12 18:41:34 [31931] pm1 lrmd: debug: operation_finished:
pgsql_monitor_4000:32721:stderr [ -- empty -- ]
Oct 12 18:41:34 [31931] pm1 lrmd: debug: operation_finished:
pgsql_monitor_4000:32721:stdout [ -- empty -- ]
Oct 12 18:41:34 [31931] pm1 lrmd: debug: log_finished: finished
- rsc:pgsql action:monitor call_id:17 pid:32721 exit-code:0
exec-time:0ms queue-time:0ms
Oct 12 18:41:35 [31929] pm1 cib: debug: cib_acl_enabled:
CIB ACL is disabled
Oct 12 18:41:35 [31929] pm1 cib: info: cib_process_ping:
Reporting our current digest to pm1: 11f884b9e61612366a27b17f6160c4e5
for 0.19.27 (0xf07600 0)
On 12.10.2015 18:06, Nikolay Popov wrote:
> Hi, Andrey.
>
> Check parameter /postgrresql.conf/ (unix_socket_directories) and
> RA pqsql (OCF_RESKEY_socketdir=/tmp)
>
> > stderr: psql: could not connect to server: No such file or directory
> > stderr: Is the server running locally and accepting
> > stderr: connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
> > stderr: WARNING: Can't get PostgreSQL recovery status. rc=2
>
> 12.10.2015 17:44, Andrey Rubik пишет:
>> Thanks for reply.
>>
>> Now I testing only Master/Slave configuration, but before i tried
>> full configuration with same results:
>>
>> pcs cluster cib pgsql_cfg
>>
>> pcs -f pgsql_cfg property set no-quorum-policy="ignore"
>> pcs -f pgsql_cfg property set stonith-enabled="false"
>> pcs -f pgsql_cfg resource defaults resource-stickiness="INFINITY"
>> pcs -f pgsql_cfg resource defaults migration-threshold="2"
>>
>> pcs -f pgsql_cfg resource create vip-master IPaddr2 \
>> ip="10.200.200.225" \
>> nic="eth0" \
>> cidr_netmask="24" \
>> op start timeout="60s" interval="0s" on-fail="restart" \
>> op monitor timeout="60s" interval="10s" on-fail="restart" \
>> op stop timeout="60s" interval="0s" on-fail="block"
>>
>> pcs -f pgsql_cfg resource create vip-rep IPaddr2 \
>> ip="10.200.200.224" \
>> nic="eth0" \
>> cidr_netmask="24" \
>> meta migration-threshold="0" \
>> op start timeout="60s" interval="0s" on-fail="stop" \
>> op monitor timeout="60s" interval="10s" on-fail="restart" \
>> op stop timeout="60s" interval="0s" on-fail="ignore"
>>
>> pcs -f pgsql_cfg resource create pgsql pgsql \
>> pgctl="/usr/pgsql-9.3/bin/pg_ctl" \
>> psql="/usr/pgsql-9.3/bin/psql" \
>> pgdata="/var/lib/pgsql/9.3/data/" \
>> tmpdir="/var/lib/pgsql/9.3/tmp/" \
>> rep_mode="sync" \
>> node_list="pg1 pg2" \
>> restore_command="cp /var/lib/pgsql/9.3/backups/%f %p" \
>> primary_conninfo_opt="keepalives_idle=60 keepalives_interval=5
>> keepalives_count=5" \
>> master_ip="10.200.200.225" \
>> restart_on_promote='false' \
>> op start timeout="60s" interval="0s" on-fail="restart" \
>> op monitor timeout="60s" interval="4s" on-fail="restart" \
>> op monitor timeout="60s" interval="3s" on-fail="restart"
>> role="Master" \
>> op promote timeout="60s" interval="0s" on-fail="restart" \
>> op demote timeout="60s" interval="0s" on-fail="stop" \
>> op stop timeout="60s" interval="0s" on-fail="block" \
>> op notify timeout="60s" interval="0s"
>>
>> pcs -f pgsql_cfg resource master msPostgresql pgsql \
>> master-max=1 master-node-max=1 clone-max=2 clone-node-max=1
>> notify=true
>>
>> pcs -f pgsql_cfg resource group add master-group vip-master vip-rep
>>
>> pcs -f pgsql_cfg constraint colocation add master-group with Master
>> msPostgresql INFINITY
>> pcs -f pgsql_cfg constraint order promote msPostgresql then start
>> master-group symmetrical=false score=INFINITY
>> pcs -f pgsql_cfg constraint order demote msPostgresql then stop
>> master-group symmetrical=false score=0
>>
>> pcs cluster cib-push pgsql_cfg
>>
>>
>>
>> crm_mon in this case:
>>
>> Stack: corosync
>> Current DC: pm2 (version 1.1.13-a14efad) - partition with quorum
>> 2 nodes and 4 resources configured
>>
>> Online: [ pm1 pm2 ]
>>
>> Full list of resources:
>>
>> Master/Slave Set: msPostgresql [pgsql]
>> Slaves: [ pm1 pm2 ]
>> Resource Group: master-group
>> vip-master (ocf::heartbeat:IPaddr2): Stopped
>> vip-rep (ocf::heartbeat:IPaddr2): Stopped
>>
>> Node Attributes:
>> * Node pm1:
>> + master-pgsql : -INFINITY
>> + pgsql-status : HS:alone
>> + pgsql-xlog-loc : 0000000000000000
>> * Node pm2:
>> + pgsql-status : HS:alone
>> + pgsql-xlog-loc : 0000000000000000
>>
>> Migration Summary:
>> * Node pm2:
>> * Node pm1:
>>
>>
>>
>>
>>
>>
>> On 12.10.2015 17:36, Nikolay Popov wrote:
>>> Hi, Andrey.
>>>
>>> You added VIP?
>>> That is a VIP (master_ip="10.200.200.233")?
>>>
>>>
>>>
>>> 12.10.2015 17:23, Andrey Rubik пишет:
>>>> Hi all.
>>>>
>>>> I'm trying to configure pgsql Master/Slave set with 2 nodes.
>>>>
>>>> I'm get successful installation Master/Slave without pacemaker.
>>>>
>>>> Pacemaker version:
>>>>
>>>> pacemakerd --features
>>>> Pacemaker 1.1.13 (Build: a14efad)
>>>> Supporting v3.0.10: generated-manpages agent-manpages ascii-docs
>>>> publican-docs ncurses libqb-logging libqb-ipc upstart systemd
>>>> nagios corosync-native atomic-attrd acls
>>>>
>>>> That's my configuration:
>>>>
>>>> pcs cluster cib pgsql_cfg
>>>>
>>>> pcs -f pgsql_cfg property set no-quorum-policy="ignore"
>>>> pcs -f pgsql_cfg property set stonith-enabled="false"
>>>>
>>>> pcs -f pgsql_cfg resource create pgsql pgsql \
>>>> pgctl="/usr/pgsql-9.3/bin/pg_ctl" \
>>>> psql="/usr/pgsql-9.3/bin/psql" \
>>>> pgdata="/var/lib/pgsql/9.3/data/" \
>>>> tmpdir="/var/lib/pgsql/9.3/tmp/" \
>>>> rep_mode="sync" \
>>>> node_list="pg1 pg2" \
>>>> restore_command="cp /var/lib/pgsql/9.3/backups/%f %p" \
>>>> primary_conninfo_opt="keepalives_idle=60 keepalives_interval=5
>>>> keepalives_count=5" \
>>>> master_ip="10.200.200.233" \
>>>> restart_on_promote='true' \
>>>> op start timeout="60s" interval="0s" on-fail="restart" \
>>>> op monitor timeout="60s" interval="4s" on-fail="restart" \
>>>> op monitor timeout="60s" interval="3s" on-fail="restart"
>>>> role="Master" \
>>>> op promote timeout="60s" interval="0s" on-fail="restart" \
>>>> op demote timeout="60s" interval="0s" on-fail="stop" \
>>>> op stop timeout="60s" interval="0s" on-fail="block" \
>>>> op notify timeout="60s" interval="0s"
>>>>
>>>> pcs -f pgsql_cfg resource master msPostgresql pgsql \
>>>> master-max=1 master-node-max=1 clone-max=2 clone-node-max=1
>>>> notify=true
>>>>
>>>> pcs cluster cib-push pgsql_cfg
>>>>
>>>>
>>>>
>>>> crm_mon:
>>>>
>>>> Stack: corosync
>>>> Current DC: pm2 (version 1.1.13-a14efad) - partition with quorum
>>>> 2 nodes and 2 resources configured
>>>>
>>>> Online: [ pm1 pm2 ]
>>>>
>>>> Full list of resources:
>>>>
>>>> Master/Slave Set: msPostgresql [pgsql]
>>>> Slaves: [ pm1 pm2 ]
>>>>
>>>> Node Attributes:
>>>> * Node pm1:
>>>> + master-pgsql : -INFINITY
>>>> + pgsql-data-status : LATEST
>>>> + pgsql-status : HS:alone
>>>> + pgsql-xlog-loc : 0000000000000000
>>>> * Node pm2:
>>>> + master-pgsql : -INFINITY
>>>> + pgsql-data-status : LATEST
>>>> + pgsql-master-baseline : 0000000000000000
>>>> + pgsql-status : HS:alone
>>>> + pgsql-xlog-loc : 0000000000000000
>>>>
>>>> Migration Summary:
>>>> * Node pm2:
>>>> * Node pm1:
>>>>
>>>>
>>>>
>>>> /var/log/pacemaker.log:
>>>>
>>>> WARNING: Can't get pg1 xlog location.
>>>> WARNING: Can't get pg2 xlog location.
>>>> WARNING: Can't get pg1 xlog location.
>>>> WARNING: Can't get pg2 xlog location.
>>>> .....
>>>>
>>>> Its very strange messages since i have pgsql-xlog-loc in crm_mon.
>>>>
>>>>
>>>>
>>>>
>>>> If I try set master manually it fails and then stop pm1 node:
>>>>
>>>> crm_resource -N pm1 -r pgsql --force-promote
>>>>
>>>> Operation promote for pgsql:0 (ocf:heartbeat:pgsql) returned 0
>>>> > stderr: INFO: Creating /var/lib/pgsql/9.3/tmp//PGSQL.lock.
>>>> > stderr: INFO: My master baseline : 0000000000000000.
>>>> > stderr: INFO: Restarting PostgreSQL instead of promote.
>>>> > stderr: INFO: waiting for server to shut down.... done server
>>>> stopped
>>>> > stderr: INFO: PostgreSQL is down
>>>> > stderr: INFO: server starting
>>>> > stderr: INFO: PostgreSQL start command sent.
>>>> > stderr: psql: could not connect to server: No such file or
>>>> directory
>>>> > stderr: Is the server running locally and accepting
>>>> > stderr: connections on Unix domain socket
>>>> "/tmp/.s.PGSQL.5432"?
>>>> > stderr: WARNING: Can't get PostgreSQL recovery status. rc=2
>>>> > stderr: WARNING: Connection error (connection to the server
>>>> went bad and the session was not interactive) occurred while
>>>> executing the psql command.
>>>> > stderr: DEBUG: PostgreSQL still hasn't started yet. Waiting...
>>>> > stderr: DEBUG: PostgreSQL is running as a primary.
>>>> > stderr: INFO: PostgreSQL is started.
>>>> > stderr: INFO: Changing pgsql-status on pm1 : HS:alone->PRI.
>>>>
>>>> crm_mon:
>>>>
>>>> Stack: corosync
>>>> Current DC: pm2 (version 1.1.13-a14efad) - partition with quorum
>>>> 2 nodes and 2 resources configured
>>>>
>>>> Online: [ pm1 pm2 ]
>>>>
>>>> Full list of resources:
>>>>
>>>> Master/Slave Set: msPostgresql [pgsql]
>>>> Slaves: [ pm2 ]
>>>> Stopped: [ pm1 ]
>>>>
>>>> Node Attributes:
>>>> * Node pm1:
>>>> + master-pgsql : -INFINITY
>>>> + master-pgsql:0 : 1000
>>>> + pgsql-data-status : LATEST
>>>> + pgsql-status : STOP
>>>> * Node pm2:
>>>> + master-pgsql : -INFINITY
>>>> + pgsql-data-status : LATEST
>>>> + pgsql-master-baseline : 0000000000000000
>>>> + pgsql-status : HS:alone
>>>> + pgsql-xlog-loc : 0000000000000000
>>>>
>>>> Migration Summary:
>>>> * Node pm2:
>>>> * Node pm1:
>>>> pgsql: migration-threshold=1000000 fail-count=1000000
>>>> last-failure='Mon Oct 12 17:12:04 2015'
>>>>
>>>> Failed Actions:
>>>> * pgsql_start_0 on pm1 'unknown error' (1): call=47,
>>>> status=complete, exitreason='My data may be inconsistent. You have
>>>> to remove /var/lib/pgsql/9.3/tmp//PGSQL.lock file to force start.',
>>>> last-rc-change='Mon Oct 12 17:12:04 2015', queued=0ms, exec=170ms
>>>>
>>>>
>>>>
>>>> That is verbose variant:
>>>> rm -f /var/lib/pgsql/9.3/tmp//PGSQL.lock
>>>> crm_resource -N pm1 -r pgsql --force-promote -V
>>>> Operation promote for pgsql:0 (ocf:heartbeat:pgsql) returned 0
>>>> > stderr: + 17:14:11: 22: '[' -x /sbin/runuser ']'
>>>> > stderr: + 17:14:11: 23: SU=runuser
>>>> > stderr: + 17:14:11: 45: OCF_RESKEY_pgctl_default=/usr/bin/pg_ctl
>>>> > stderr: + 17:14:11: 46: OCF_RESKEY_psql_default=/usr/bin/psql
>>>> > stderr: + 17:14:11: 47:
>>>> OCF_RESKEY_pgdata_default=/var/lib/pgsql/data
>>>> > stderr: + 17:14:11: 48: OCF_RESKEY_pgdba_default=postgres
>>>> > stderr: + 17:14:11: 49: OCF_RESKEY_pghost_default=
>>>> > stderr: + 17:14:11: 50: OCF_RESKEY_pgport_default=5432
>>>> > stderr: + 17:14:11: 51: OCF_RESKEY_pglibs_default=/usr/lib
>>>> > stderr: + 17:14:11: 52: OCF_RESKEY_start_opt_default=
>>>> > stderr: + 17:14:11: 53: OCF_RESKEY_pgdb_default=template1
>>>> > stderr: + 17:14:11: 54: OCF_RESKEY_logfile_default=/dev/null
>>>> > stderr: + 17:14:11: 55: OCF_RESKEY_stop_escalate_default=30
>>>> > stderr: + 17:14:11: 56: OCF_RESKEY_monitor_user_default=
>>>> > stderr: + 17:14:11: 57: OCF_RESKEY_monitor_password_default=
>>>> > stderr: + 17:14:11: 58: OCF_RESKEY_monitor_sql_default='select
>>>> now();'
>>>> > stderr: + 17:14:11: 59:
>>>> OCF_RESKEY_check_wal_receiver_default=false
>>>> > stderr: + 17:14:11: 61: OCF_RESKEY_rep_mode_default=none
>>>> > stderr: + 17:14:11: 62: OCF_RESKEY_node_list_default=
>>>> > stderr: + 17:14:11: 63: OCF_RESKEY_restore_command_default=
>>>> > stderr: + 17:14:11: 64:
>>>> OCF_RESKEY_archive_cleanup_command_default=
>>>> > stderr: + 17:14:11: 65: OCF_RESKEY_recovery_end_command_default=
>>>> > stderr: + 17:14:11: 66: OCF_RESKEY_master_ip_default=
>>>> > stderr: + 17:14:11: 67: OCF_RESKEY_repuser_default=postgres
>>>> > stderr: + 17:14:11: 68: OCF_RESKEY_primary_conninfo_opt_default=
>>>> > stderr: + 17:14:11: 69:
>>>> OCF_RESKEY_restart_on_promote_default=false
>>>> > stderr: + 17:14:11: 70:
>>>> OCF_RESKEY_tmpdir_default=/var/lib/pgsql/tmp
>>>> > stderr: + 17:14:11: 71: OCF_RESKEY_xlog_check_count_default=3
>>>> > stderr: + 17:14:11: 72: OCF_RESKEY_crm_attr_timeout_default=5
>>>> > stderr: + 17:14:11: 73:
>>>> OCF_RESKEY_stop_escalate_in_slave_default=30
>>>> > stderr: + 17:14:11: 74: OCF_RESKEY_replication_slot_name_default=
>>>> > stderr: + 17:14:11: 76: : /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: + 17:14:11: 77: : /usr/pgsql-9.3/bin/psql
>>>> > stderr: + 17:14:11: 78: : /var/lib/pgsql/9.3/data/
>>>> > stderr: + 17:14:11: 79: : postgres
>>>> > stderr: + 17:14:11: 80: :
>>>> > stderr: + 17:14:11: 81: : 5432
>>>> > stderr: + 17:14:11: 82: : /usr/lib
>>>> > stderr: + 17:14:11: 83: : /var/lib/pgsql/9.3/data//postgresql.conf
>>>> > stderr: + 17:14:11: 84: :
>>>> > stderr: + 17:14:11: 85: : template1
>>>> > stderr: + 17:14:11: 86: : /dev/null
>>>> > stderr: + 17:14:11: 87: : 30
>>>> > stderr: + 17:14:11: 88: :
>>>> > stderr: + 17:14:11: 89: :
>>>> > stderr: + 17:14:11: 90: : select 'now();'
>>>> > stderr: + 17:14:11: 91: : false
>>>> > stderr: + 17:14:11: 94: : sync
>>>> > stderr: + 17:14:11: 95: : pg1 pg2
>>>> > stderr: + 17:14:11: 96: : cp /var/lib/pgsql/9.3/backups/%f %p
>>>> > stderr: + 17:14:11: 97: :
>>>> > stderr: + 17:14:11: 98: :
>>>> > stderr: + 17:14:11: 99: : 10.200.200.233
>>>> > stderr: + 17:14:11: 100: : postgres
>>>> > stderr: + 17:14:11: 101: : keepalives_idle=60
>>>> keepalives_interval=5 keepalives_count=5
>>>> > stderr: + 17:14:11: 102: : true
>>>> > stderr: + 17:14:11: 103: : /var/lib/pgsql/9.3/tmp/
>>>> > stderr: + 17:14:11: 104: : 3
>>>> > stderr: + 17:14:11: 105: : 5
>>>> > stderr: + 17:14:11: 106: : 30
>>>> > stderr: + 17:14:11: 107: :
>>>> > stderr: + 17:14:11: 1935: '[' 1 -ne 1 ']'
>>>> > stderr: + 17:14:11: 1942:
>>>> PIDFILE=/var/lib/pgsql/9.3/data//postmaster.pid
>>>> > stderr: + 17:14:11: 1943:
>>>> BACKUPLABEL=/var/lib/pgsql/9.3/data//backup_label
>>>> > stderr: ++ 17:14:11: 1944: echo pgsql:0
>>>> > stderr: ++ 17:14:11: 1944: cut -d : -f 1
>>>> > stderr: + 17:14:11: 1944: RESOURCE_NAME=pgsql
>>>> > stderr: + 17:14:11: 1945:
>>>> PGSQL_WAL_RECEIVER_STATUS_ATTR=pgsql-receiver-status
>>>> > stderr: + 17:14:11: 1946:
>>>> RECOVERY_CONF=/var/lib/pgsql/9.3/data//recovery.conf
>>>> > stderr: ++ 17:14:11: 1947: ocf_local_nodename
>>>> > stderr: ++ 17:14:11: 1947: tr '[A-Z]' '[a-z]'
>>>> > stderr: ++ 17:14:11: ocf_local_nodename:570: which pacemakerd
>>>> > stderr: ++ 17:14:11: ocf_local_nodename:571: '[' 0 -eq 0 ']'
>>>> > stderr: +++ 17:14:11: ocf_local_nodename:572: pacemakerd '-$'
>>>> > stderr: +++ 17:14:12: ocf_local_nodename:572: grep 'Pacemaker .*'
>>>> > stderr: +++ 17:14:12: ocf_local_nodename:572: awk '{ print $2 }'
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:572: local version=1.1.13
>>>> > stderr: +++ 17:14:12: ocf_local_nodename:573: echo 1.1.13
>>>> > stderr: +++ 17:14:12: ocf_local_nodename:573: awk -F- '{ print
>>>> $1 }'
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:573: version=1.1.13
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:574: ocf_version_cmp
>>>> 1.1.13 1.1.8
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:543: ocf_is_ver 1.1.13
>>>> > stderr: ++ 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>>>> > stderr: ++ 17:14:12: ocf_is_ver:513: echo 1.1.13
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:544: ocf_is_ver 1.1.8
>>>> > stderr: ++ 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>>>> > stderr: ++ 17:14:12: ocf_is_ver:513: echo 1.1.8
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:545: local v1=1.1.13
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:546: local v2=1.1.8
>>>> > stderr: +++ 17:14:12: ocf_version_cmp:547: ocf_ver_level 1.1.13
>>>> > stderr: +++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>>>> > stderr: +++ 17:14:12: ocf_ver_level:521: echo 1.1.13
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:547: local v1_level=3
>>>> > stderr: +++ 17:14:12: ocf_version_cmp:548: ocf_ver_level 1.1.8
>>>> > stderr: +++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>>>> > stderr: +++ 17:14:12: ocf_ver_level:521: echo 1.1.8
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:548: local v2_level=3
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:549: local level_diff
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:550: '[' 3 -lt 3 ']'
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:553: '[' 3 -gt 3 ']'
>>>> > stderr: +++ 17:14:12: ocf_version_cmp:557: ocf_ver2num 1.1.13
>>>> > stderr: +++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>>>> > stderr: {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>>>> > stderr: '
>>>> > stderr: +++ 17:14:12: ocf_ver2num:516: echo 1.1.13
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:557: v1=1001013
>>>> > stderr: +++ 17:14:12: ocf_version_cmp:558: ocf_ver2num 1.1.8
>>>> > stderr: +++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>>>> > stderr: {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>>>> > stderr: '
>>>> > stderr: +++ 17:14:12: ocf_ver2num:516: echo 1.1.8
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:558: v2=1001008
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:559: '[' 1001013 -eq
>>>> 1001008 ']'
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:561: '[' 1001013 -lt
>>>> 1001008 ']'
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:564: return 2
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:575: '[' 2 -eq 2 ']'
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:576: which crm_node
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:577: '[' 0 -eq 0 ']'
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:578: crm_node -n
>>>> > stderr: ++ 17:14:12: ocf_local_nodename:579: return
>>>> > stderr: + 17:14:12: 1947: NODENAME=pm1
>>>> > stderr: + 17:14:12: 1949: is_replication
>>>> > stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a
>>>> sync '!=' slave ']'
>>>> > stderr: + 17:14:12: is_replication:1288: return 0
>>>> > stderr: + 17:14:12: 1950:
>>>> REP_MODE_CONF=/var/lib/pgsql/9.3/tmp//rep_mode.conf
>>>> > stderr: + 17:14:12: 1951:
>>>> PGSQL_LOCK=/var/lib/pgsql/9.3/tmp//PGSQL.lock
>>>> > stderr: + 17:14:12: 1952:
>>>> XLOG_NOTE_FILE=/var/lib/pgsql/9.3/tmp//xlog_note
>>>> > stderr: + 17:14:12: 1954: CRM_MASTER='/usr/sbin/crm_master -l
>>>> reboot'
>>>> > stderr: + 17:14:12: 1955:
>>>> CRM_ATTR_REBOOT='/usr/sbin/crm_attribute -l reboot'
>>>> > stderr: + 17:14:12: 1956:
>>>> CRM_ATTR_FOREVER='/usr/sbin/crm_attribute -l forever'
>>>> > stderr: + 17:14:12: 1957: CRM_FAILCOUNT=/usr/sbin/crm_failcount
>>>> > stderr: + 17:14:12: 1959: CAN_NOT_PROMOTE=-INFINITY
>>>> > stderr: + 17:14:12: 1960: CAN_PROMOTE=100
>>>> > stderr: + 17:14:12: 1961: PROMOTE_ME=1000
>>>> > stderr: + 17:14:12: 1963: CHECK_MS_SQL='select
>>>> pg_is_in_recovery()'
>>>> > stderr: + 17:14:12: 1964: CHECK_XLOG_LOC_SQL='select
>>>> pg_last_xlog_replay_location(),pg_last_xlog_receive_location()'
>>>> > stderr: + 17:14:12: 1965: CHECK_REPLICATION_STATE_SQL='select
>>>> application_name,upper(state),upper(sync_state) from
>>>> pg_stat_replication'
>>>> > stderr: + 17:14:12: 1967: PGSQL_STATUS_ATTR=pgsql-status
>>>> > stderr: + 17:14:12: 1968: PGSQL_DATA_STATUS_ATTR=pgsql-data-status
>>>> > stderr: + 17:14:12: 1969: PGSQL_XLOG_LOC_NAME=pgsql-xlog-loc
>>>> > stderr: + 17:14:12: 1970:
>>>> PGSQL_MASTER_BASELINE=pgsql-master-baseline
>>>> > stderr: ++ 17:14:12: 1972: echo pg1 pg2
>>>> > stderr: ++ 17:14:12: 1972: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:12: 1972: NODE_LIST='pg1 pg2'
>>>> > stderr: + 17:14:12: 1973: RE_CONTROL_SLAVE=false
>>>> > stderr: + 17:14:12: 1976: case "$1" in
>>>> > stderr: + 17:14:12: 1984: pgsql_validate_all
>>>> > stderr: + 17:14:12: pgsql_validate_all:1749: local version
>>>> > stderr: + 17:14:12: pgsql_validate_all:1750: local check_config_rc
>>>> > stderr: + 17:14:12: pgsql_validate_all:1751: local rep_mode_string
>>>> > stderr: + 17:14:12: pgsql_validate_all:1752: local
>>>> socket_directories
>>>> > stderr: ++ 17:14:12: pgsql_validate_all:1754: cat
>>>> /var/lib/pgsql/9.3/data//PG_VERSION
>>>> > stderr: + 17:14:12: pgsql_validate_all:1754: version=9.3
>>>> > stderr: + 17:14:12: pgsql_validate_all:1756: check_binary2
>>>> /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: + 17:14:12: check_binary2:1724: have_binary
>>>> /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: + 17:14:12: have_binary:68: '[' '' = 1 ']'
>>>> > stderr: ++ 17:14:12: have_binary:71: echo
>>>> /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: ++ 17:14:12: have_binary:71: sed -e 's/ -.*//'
>>>> > stderr: + 17:14:12: have_binary:71: local
>>>> bin=/usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: ++ 17:14:12: have_binary:72: which
>>>> /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: + 17:14:12: have_binary:72: test -x
>>>> /usr/pgsql-9.3/bin/pg_ctl
>>>> > stderr: + 17:14:12: check_binary2:1728: return 0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1757: check_binary2
>>>> /usr/pgsql-9.3/bin/psql
>>>> > stderr: + 17:14:12: check_binary2:1724: have_binary
>>>> /usr/pgsql-9.3/bin/psql
>>>> > stderr: + 17:14:12: have_binary:68: '[' '' = 1 ']'
>>>> > stderr: ++ 17:14:12: have_binary:71: echo /usr/pgsql-9.3/bin/psql
>>>> > stderr: ++ 17:14:12: have_binary:71: sed -e 's/ -.*//'
>>>> > stderr: + 17:14:12: have_binary:71: local
>>>> bin=/usr/pgsql-9.3/bin/psql
>>>> > stderr: ++ 17:14:12: have_binary:72: which /usr/pgsql-9.3/bin/psql
>>>> > stderr: + 17:14:12: have_binary:72: test -x
>>>> /usr/pgsql-9.3/bin/psql
>>>> > stderr: + 17:14:12: check_binary2:1728: return 0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1761: check_config
>>>> /var/lib/pgsql/9.3/data//postgresql.conf
>>>> > stderr: + 17:14:12: check_config:1732: local rc=0
>>>> > stderr: + 17:14:12: check_config:1734: '[' '!' -f
>>>> /var/lib/pgsql/9.3/data//postgresql.conf ']'
>>>> > stderr: + 17:14:12: check_config:1744: return 0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1762: check_config_rc=0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1763: '[' 0 -eq 2 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1764: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1765: ocf_version_cmp
>>>> 9.3 9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:543: ocf_is_ver 9.3
>>>> > stderr: + 17:14:12: ocf_is_ver:513: echo 9.3
>>>> > stderr: + 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>>>> > stderr: + 17:14:12: ocf_version_cmp:544: ocf_is_ver 9.3
>>>> > stderr: + 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>>>> > stderr: + 17:14:12: ocf_is_ver:513: echo 9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:545: local v1=9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:546: local v2=9.3
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:547: ocf_ver_level 9.3
>>>> > stderr: ++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>>>> > stderr: ++ 17:14:12: ocf_ver_level:521: echo 9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:547: local v1_level=2
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:548: ocf_ver_level 9.3
>>>> > stderr: ++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>>>> > stderr: ++ 17:14:12: ocf_ver_level:521: echo 9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:548: local v2_level=2
>>>> > stderr: + 17:14:12: ocf_version_cmp:549: local level_diff
>>>> > stderr: + 17:14:12: ocf_version_cmp:550: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:12: ocf_version_cmp:553: '[' 2 -gt 2 ']'
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:557: ocf_ver2num 9.3
>>>> > stderr: ++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>>>> > stderr: {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>>>> > stderr: '
>>>> > stderr: ++ 17:14:12: ocf_ver2num:516: echo 9.3
>>>> > stderr: + 17:14:12: ocf_version_cmp:557: v1=9003
>>>> > stderr: ++ 17:14:12: ocf_version_cmp:558: ocf_ver2num 9.3
>>>> > stderr: ++ 17:14:12: ocf_ver2num:516: echo 9.3
>>>> > stderr: ++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>>>> > stderr: {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>>>> > stderr: '
>>>> > stderr: + 17:14:12: ocf_version_cmp:558: v2=9003
>>>> > stderr: + 17:14:12: ocf_version_cmp:559: '[' 9003 -eq 9003 ']'
>>>> > stderr: + 17:14:12: ocf_version_cmp:560: return 1
>>>> > stderr: + 17:14:12: pgsql_validate_all:1766: '[' 1 -eq 0 ']'
>>>> > stderr: ++ 17:14:12: pgsql_validate_all:1770: get_pgsql_param
>>>> unix_socket_directories
>>>> > stderr: ++ 17:14:12: get_pgsql_param:32: local param_name
>>>> > stderr: ++ 17:14:12: get_pgsql_param:34:
>>>> param_name=unix_socket_directories
>>>> > stderr: ++ 17:14:12: get_pgsql_param:39: perl_code='if
>>>> (/^\s*unix_socket_directories[\s=]+\s*(.*)$/) {
>>>> > stderr: $dir=$1;
>>>> > stderr: $dir =~ s/\s*\#.*//;
>>>> > stderr: $dir =~ s/^'\''(\S*)'\''/$1/;
>>>> > stderr: print $dir;}'
>>>> > stderr: ++ 17:14:12: get_pgsql_param:41: perl -ne 'if
>>>> (/^\s*unix_socket_directories[\s=]+\s*(.*)$/) {
>>>> > stderr: $dir=$1;
>>>> > stderr: $dir =~ s/\s*\#.*//;
>>>> > stderr: $dir =~ s/^'\''(\S*)'\''/$1/;
>>>> > stderr: print $dir;}'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1770: socket_directories=
>>>> > stderr: + 17:14:12: pgsql_validate_all:1771: '[' -n '' ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1782: getent passwd
>>>> postgres
>>>> > stderr: + 17:14:12: pgsql_validate_all:1783: '[' '!' 0 -eq 0 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1788: ocf_is_probe
>>>> > stderr: + 17:14:12: ocf_is_probe:490: '[' promote = monitor -a
>>>> 0 = 0 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1791: runasowner 'test
>>>> -w /var/lib/pgsql/9.3/data/'
>>>> > stderr: + 17:14:12: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:12: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:12: runasowner:469: local var
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; test -w
>>>> /var/lib/pgsql/9.3/data/'
>>>> > stderr: + 17:14:12: ocf_run:413: local rc
>>>> > stderr: + 17:14:12: ocf_run:414: local output
>>>> > stderr: + 17:14:12: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:12: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:417: local var
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:426: echo -err
>>>> > stderr: ++ 17:14:12: ocf_run:426: sed -e s/-//g
>>>> > stderr: + 17:14:12: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:427: shift 1
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; test -w /var/lib/pgsql/9.3/data/'
>>>> > stderr: + 17:14:12: ocf_run:433: output=
>>>> > stderr: + 17:14:12: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:12: ocf_run:435: echo
>>>> > stderr: + 17:14:12: ocf_run:435: output=
>>>> > stderr: + 17:14:12: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:12: ocf_run:437: '[' 1 -a '!' -z '' ']'
>>>> > stderr: + 17:14:12: ocf_run:440: return 0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1797: '[' -n '' -a '!'
>>>> -n '' ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1803: '[' '!' -n '' -a
>>>> -n '' ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1809: is_replication
>>>> > stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a
>>>> sync '!=' slave ']'
>>>> > stderr: + 17:14:12: is_replication:1288: return 0
>>>> > stderr: ++ 17:14:12: pgsql_validate_all:1810: printf '9.3\n9.1'
>>>> > stderr: ++ 17:14:12: pgsql_validate_all:1810: sort -n
>>>> > stderr: ++ 17:14:12: pgsql_validate_all:1810: head -1
>>>> > stderr: + 17:14:12: pgsql_validate_all:1810: '[' 9.1 '!=' 9.1 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1814: '[' '!' -n
>>>> 10.200.200.233 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1820: is_replication
>>>> > stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a
>>>> sync '!=' slave ']'
>>>> > stderr: + 17:14:12: is_replication:1288: return 0
>>>> > stderr: + 17:14:12: pgsql_validate_all:1821: ocf_is_ms
>>>> > stderr: + 17:14:12: ocf_is_ms:504: '[' '!' -z 1 ']'
>>>> > stderr: + 17:14:12: ocf_is_ms:504: '[' 1 -gt 0 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1825: '[' '!' sync =
>>>> sync -a '!' sync = async ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1829: '[' '!' -n 'pg1
>>>> pg2' ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1833: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1834:
>>>> rep_mode_string='include
>>>> '\''/var/lib/pgsql/9.3/tmp//rep_mode.conf'\'' # added by pgsql RA'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1835: '[' sync = sync ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1836: grep -q 'include
>>>> '\''/var/lib/pgsql/9.3/tmp//rep_mode.conf'\'' # added by pgsql RA'
>>>> /var/lib/pgsql/9.3/data//postgresql.conf
>>>> > stderr: + 17:14:12: pgsql_validate_all:1848: mkdir -p
>>>> /var/lib/pgsql/9.3/tmp/
>>>> > stderr: + 17:14:12: pgsql_validate_all:1848: chown postgres
>>>> /var/lib/pgsql/9.3/tmp/
>>>> > stderr: + 17:14:12: pgsql_validate_all:1848: chmod 700
>>>> /var/lib/pgsql/9.3/tmp/
>>>> > stderr: + 17:14:12: pgsql_validate_all:1854: '[' sync = slave ']'
>>>> > stderr: + 17:14:12: pgsql_validate_all:1861: use_replication_slot
>>>> > stderr: + 17:14:12: use_replication_slot:1294: '[' -n '' ']'
>>>> > stderr: + 17:14:12: use_replication_slot:1298: return 1
>>>> > stderr: + 17:14:12: pgsql_validate_all:1869: return 0
>>>> > stderr: + 17:14:12: 1985: rc=0
>>>> > stderr: + 17:14:12: 1987: '[' promote = validate-all ']'
>>>> > stderr: + 17:14:12: 1989: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: 2002: id -u -n
>>>> > stderr: + 17:14:12: 2002: US=root
>>>> > stderr: + 17:14:12: 2004: '[' root '!=' root -a root '!='
>>>> postgres ']'
>>>> > stderr: + 17:14:12: 2011: '[' -n '' ']'
>>>> > stderr: + 17:14:12: 2016: psql_options='-p 5432 -U postgres
>>>> template1'
>>>> > stderr: + 17:14:12: 2019: '[' -n '' ']'
>>>> > stderr: + 17:14:12: 2022: '[' -n '' ']'
>>>> > stderr: + 17:14:12: 2027: '[' -n 5432 ']'
>>>> > stderr: + 17:14:12: 2028: export PGPORT=5432
>>>> > stderr: + 17:14:12: 2028: PGPORT=5432
>>>> > stderr: + 17:14:12: 2031: '[' -n /usr/lib ']'
>>>> > stderr: + 17:14:12: 2032: '[' -n '' ']'
>>>> > stderr: + 17:14:12: 2035: export LD_LIBRARY_PATH=/usr/lib
>>>> > stderr: + 17:14:12: 2035: LD_LIBRARY_PATH=/usr/lib
>>>> > stderr: + 17:14:12: 2041: case "$1" in
>>>> > stderr: + 17:14:12: 2057: pgsql_promote
>>>> > stderr: + 17:14:12: pgsql_promote:665: local target
>>>> > stderr: + 17:14:12: pgsql_promote:666: local rc
>>>> > stderr: + 17:14:12: pgsql_promote:668: is_replication
>>>> > stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a
>>>> sync '!=' slave ']'
>>>> > stderr: + 17:14:12: is_replication:1288: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:672: rm -f
>>>> '/var/lib/pgsql/9.3/tmp//xlog_note.*'
>>>> > stderr: + 17:14:12: pgsql_promote:674: for target in '$NODE_LIST'
>>>> > stderr: + 17:14:12: pgsql_promote:675: '[' pg1 = pm1 ']'
>>>> > stderr: + 17:14:12: pgsql_promote:676: change_data_status pg1
>>>> DISCONNECT
>>>> > stderr: + 17:14:12: change_data_status:1569: local output
>>>> > stderr: + 17:14:12: change_data_status:1571: node_exist pg1
>>>> > stderr: + 17:14:12: node_exist:1720: print_crm_mon
>>>> > stderr: + 17:14:12: node_exist:1720: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:12: node_exist:1720: grep -q '^node pg1'
>>>> > stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:12: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: + 17:14:12: change_data_status:1572: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:677: change_master_score pg1
>>>> -INFINITY
>>>> > stderr: + 17:14:12: change_master_score:1604: local instance
>>>> > stderr: + 17:14:12: change_master_score:1606: is_node_online pg1
>>>> > stderr: + 17:14:12: is_node_online:1716: print_crm_mon
>>>> > stderr: + 17:14:12: is_node_online:1716: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:12: is_node_online:1716: grep -e '^node pg1 '
>>>> -e '^node pg1:'
>>>> > stderr: + 17:14:12: is_node_online:1716: grep -q -v offline
>>>> > stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:12: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: + 17:14:12: change_master_score:1607: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:674: for target in '$NODE_LIST'
>>>> > stderr: + 17:14:12: pgsql_promote:675: '[' pg2 = pm1 ']'
>>>> > stderr: + 17:14:12: pgsql_promote:676: change_data_status pg2
>>>> DISCONNECT
>>>> > stderr: + 17:14:12: change_data_status:1569: local output
>>>> > stderr: + 17:14:12: change_data_status:1571: node_exist pg2
>>>> > stderr: + 17:14:12: node_exist:1720: print_crm_mon
>>>> > stderr: + 17:14:12: node_exist:1720: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:12: node_exist:1720: grep -q '^node pg2'
>>>> > stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:12: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: + 17:14:12: change_data_status:1572: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:677: change_master_score pg2
>>>> -INFINITY
>>>> > stderr: + 17:14:12: change_master_score:1604: local instance
>>>> > stderr: + 17:14:12: change_master_score:1606: is_node_online pg2
>>>> > stderr: + 17:14:12: is_node_online:1716: grep -q -v offline
>>>> > stderr: + 17:14:12: is_node_online:1716: print_crm_mon
>>>> > stderr: + 17:14:12: is_node_online:1716: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:12: is_node_online:1716: grep -e '^node pg2 '
>>>> -e '^node pg2:'
>>>> > stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:12: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:12: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:12 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: + 17:14:12: change_master_score:1607: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:680: ocf_log info 'Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>>>> > stderr: + 17:14:12: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:12: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:12: ocf_log:326: shift
>>>> > stderr: + 17:14:12: ocf_log:327: __OCF_MSG='Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>>>> > stderr: + 17:14:12: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:12: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:12: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:12: ocf_log:341: ha_log 'INFO: Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>>>> > stderr: + 17:14:12: ha_log:253: __ha_log 'INFO: Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>>>> > stderr: + 17:14:12: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:12: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:12: __ha_log:188: '[' 'xINFO: Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:12: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:12: __ha_log:192: tty
>>>> > stderr: + 17:14:12: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:12: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:12: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:12: __ha_log:202: echo 'INFO: Creating
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>>>> > stderr: INFO: Creating /var/lib/pgsql/9.3/tmp//PGSQL.lock.
>>>> > stderr: + 17:14:12: __ha_log:204: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:681: touch
>>>> /var/lib/pgsql/9.3/tmp//PGSQL.lock
>>>> > stderr: + 17:14:12: pgsql_promote:682: show_master_baseline
>>>> > stderr: + 17:14:12: show_master_baseline:1438: local rc
>>>> > stderr: + 17:14:12: show_master_baseline:1439: local location
>>>> > stderr: ++ 17:14:12: show_master_baseline:1441: get_my_location
>>>> > stderr: ++ 17:14:12: get_my_location:1389: local rc
>>>> > stderr: ++ 17:14:12: get_my_location:1390: local output
>>>> > stderr: ++ 17:14:12: get_my_location:1391: local replay_loc
>>>> > stderr: ++ 17:14:12: get_my_location:1392: local receive_loc
>>>> > stderr: ++ 17:14:12: get_my_location:1393: local output1
>>>> > stderr: ++ 17:14:12: get_my_location:1394: local output2
>>>> > stderr: ++ 17:14:12: get_my_location:1395: local log1
>>>> > stderr: ++ 17:14:12: get_my_location:1396: local log2
>>>> > stderr: ++ 17:14:12: get_my_location:1397: local newer_location
>>>> > stderr: +++ 17:14:12: get_my_location:1399: exec_sql ''
>>>> > stderr: +++ 17:14:12: exec_sql:518: local sql=
>>>> > stderr: +++ 17:14:12: exec_sql:519: local output
>>>> > stderr: +++ 17:14:12: exec_sql:520: local rc
>>>> > stderr: ++++ 17:14:12: exec_sql:524: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; /usr/pgsql-9.3/bin/psql -p 5432 -U
>>>> postgres template1 -U postgres -Atc ""'
>>>> > stderr: +++ 17:14:12: exec_sql:524: output=
>>>> > stderr: +++ 17:14:12: exec_sql:525: rc=0
>>>> > stderr: +++ 17:14:12: exec_sql:527: echo
>>>> > stderr: +++ 17:14:12: exec_sql:528: return 0
>>>> > stderr: ++ 17:14:12: get_my_location:1399: output=
>>>> > stderr: ++ 17:14:12: get_my_location:1400: rc=0
>>>> > stderr: ++ 17:14:12: get_my_location:1402: '[' 0 -ne 0 ']'
>>>> > stderr: +++ 17:14:12: get_my_location:1406: cut -d '|' -f 1
>>>> > stderr: +++ 17:14:12: get_my_location:1406: echo
>>>> > stderr: ++ 17:14:12: get_my_location:1406: replay_loc=
>>>> > stderr: +++ 17:14:12: get_my_location:1407: cut -d '|' -f 2
>>>> > stderr: +++ 17:14:12: get_my_location:1407: echo
>>>> > stderr: ++ 17:14:12: get_my_location:1407: receive_loc=
>>>> > stderr: +++ 17:14:12: get_my_location:1409: echo ''
>>>> > stderr: +++ 17:14:12: get_my_location:1409: cut -d / -f 1
>>>> > stderr: ++ 17:14:12: get_my_location:1409: output1=
>>>> > stderr: +++ 17:14:12: get_my_location:1410: echo ''
>>>> > stderr: +++ 17:14:12: get_my_location:1410: cut -d / -f 2
>>>> > stderr: ++ 17:14:12: get_my_location:1410: output2=
>>>> > stderr: +++ 17:14:12: get_my_location:1411: printf '%08s\n'
>>>> > stderr: +++ 17:14:12: get_my_location:1411: sed 's/ /0/g'
>>>> > stderr: ++ 17:14:12: get_my_location:1411: log1=00000000
>>>> > stderr: +++ 17:14:12: get_my_location:1412: printf '%08s\n'
>>>> > stderr: +++ 17:14:12: get_my_location:1412: sed 's/ /0/g'
>>>> > stderr: ++ 17:14:12: get_my_location:1412: log2=00000000
>>>> > stderr: ++ 17:14:12: get_my_location:1413:
>>>> replay_loc=0000000000000000
>>>> > stderr: +++ 17:14:12: get_my_location:1415: cut -d / -f 1
>>>> > stderr: +++ 17:14:12: get_my_location:1415: echo ''
>>>> > stderr: ++ 17:14:12: get_my_location:1415: output1=
>>>> > stderr: +++ 17:14:12: get_my_location:1416: cut -d / -f 2
>>>> > stderr: +++ 17:14:12: get_my_location:1416: echo ''
>>>> > stderr: ++ 17:14:12: get_my_location:1416: output2=
>>>> > stderr: +++ 17:14:12: get_my_location:1417: sed 's/ /0/g'
>>>> > stderr: +++ 17:14:12: get_my_location:1417: printf '%08s\n'
>>>> > stderr: ++ 17:14:12: get_my_location:1417: log1=00000000
>>>> > stderr: +++ 17:14:12: get_my_location:1418: sed 's/ /0/g'
>>>> > stderr: +++ 17:14:12: get_my_location:1418: printf '%08s\n'
>>>> > stderr: ++ 17:14:12: get_my_location:1418: log2=00000000
>>>> > stderr: ++ 17:14:12: get_my_location:1419:
>>>> receive_loc=0000000000000000
>>>> > stderr: +++ 17:14:12: get_my_location:1421: printf
>>>> '0000000000000000\n0000000000000000'
>>>> > stderr: +++ 17:14:12: get_my_location:1421: sort -r
>>>> > stderr: +++ 17:14:12: get_my_location:1421: head -1
>>>> > stderr: ++ 17:14:12: get_my_location:1421:
>>>> newer_location=0000000000000000
>>>> > stderr: ++ 17:14:12: get_my_location:1422: echo 0000000000000000
>>>> > stderr: ++ 17:14:12: get_my_location:1423: return 0
>>>> > stderr: + 17:14:12: show_master_baseline:1441:
>>>> location=0000000000000000
>>>> > stderr: + 17:14:12: show_master_baseline:1442: ocf_log info 'My
>>>> master baseline : 0000000000000000.'
>>>> > stderr: + 17:14:12: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:12: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:12: ocf_log:326: shift
>>>> > stderr: + 17:14:12: ocf_log:327: __OCF_MSG='My master baseline
>>>> : 0000000000000000.'
>>>> > stderr: + 17:14:12: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:12: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:12: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:12: ocf_log:341: ha_log 'INFO: My master
>>>> baseline : 0000000000000000.'
>>>> > stderr: + 17:14:12: ha_log:253: __ha_log 'INFO: My master
>>>> baseline : 0000000000000000.'
>>>> > stderr: + 17:14:12: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:12: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:12: __ha_log:188: '[' 'xINFO: My master
>>>> baseline : 0000000000000000.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:12: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:12: __ha_log:192: tty
>>>> > stderr: + 17:14:12: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:12: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:12: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:12: __ha_log:202: echo 'INFO: My master
>>>> baseline : 0000000000000000.'
>>>> > stderr: INFO: My master baseline : 0000000000000000.
>>>> > stderr: + 17:14:12: __ha_log:204: return 0
>>>> > stderr: + 17:14:12: show_master_baseline:1443: exec_with_retry
>>>> 0 /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-master-baseline
>>>> -v 0000000000000000
>>>> > stderr: + 17:14:12: exec_with_retry:1689: local count=86400
>>>> > stderr: + 17:14:12: exec_with_retry:1690: local output
>>>> > stderr: + 17:14:12: exec_with_retry:1691: local rc
>>>> > stderr: + 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:12: exec_with_retry:1696: shift
>>>> > stderr: + 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: ++ 17:14:12: exec_with_retry:1699:
>>>> /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-master-baseline
>>>> -v 0000000000000000
>>>> > stderr: + 17:14:12: exec_with_retry:1699: output=
>>>> > stderr: + 17:14:12: exec_with_retry:1700: rc=0
>>>> > stderr: + 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:12: exec_with_retry:1706: printf ''
>>>> > stderr: + 17:14:12: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:684: ocf_is_true true
>>>> > stderr: + 17:14:12: ocf_is_true:101: case "$1" in
>>>> > stderr: + 17:14:12: ocf_is_true:102: true
>>>> > stderr: + 17:14:12: pgsql_promote:685: ocf_log info 'Restarting
>>>> PostgreSQL instead of promote.'
>>>> > stderr: + 17:14:12: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:12: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:12: ocf_log:326: shift
>>>> > stderr: + 17:14:12: ocf_log:327: __OCF_MSG='Restarting
>>>> PostgreSQL instead of promote.'
>>>> > stderr: + 17:14:12: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:12: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:12: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:12: ocf_log:341: ha_log 'INFO: Restarting
>>>> PostgreSQL instead of promote.'
>>>> > stderr: + 17:14:12: ha_log:253: __ha_log 'INFO: Restarting
>>>> PostgreSQL instead of promote.'
>>>> > stderr: + 17:14:12: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:12: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:12: __ha_log:188: '[' 'xINFO: Restarting
>>>> PostgreSQL instead of promote.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:12: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:12: __ha_log:192: tty
>>>> > stderr: + 17:14:12: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:12: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:12: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:12: __ha_log:202: echo 'INFO: Restarting
>>>> PostgreSQL instead of promote.'
>>>> > stderr: INFO: Restarting PostgreSQL instead of promote.
>>>> > stderr: + 17:14:12: __ha_log:204: return 0
>>>> > stderr: + 17:14:12: pgsql_promote:687: pgsql_real_stop slave
>>>> > stderr: + 17:14:12: pgsql_real_stop:758: local rc
>>>> > stderr: + 17:14:12: pgsql_real_stop:759: local count
>>>> > stderr: + 17:14:12: pgsql_real_stop:760: local stop_escalate
>>>> > stderr: + 17:14:12: pgsql_real_stop:762: ocf_is_true false
>>>> > stderr: + 17:14:12: ocf_is_true:101: case "$1" in
>>>> > stderr: + 17:14:12: ocf_is_true:103: false
>>>> > stderr: + 17:14:12: pgsql_real_stop:766: pgsql_status
>>>> > stderr: + 17:14:12: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: ++ 17:14:12: pgsql_status:868: head -n 1
>>>> /var/lib/pgsql/9.3/data//postmaster.pid
>>>> > stderr: + 17:14:12: pgsql_status:868: PID=6046
>>>> > stderr: + 17:14:12: pgsql_status:869: runasowner 'kill -s 0
>>>> 6046 >/dev/null 2>&1'
>>>> > stderr: + 17:14:12: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:12: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:12: runasowner:469: local var
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; kill -s 0 6046 >/dev/null
>>>> 2>&1'
>>>> > stderr: + 17:14:12: ocf_run:413: local rc
>>>> > stderr: + 17:14:12: ocf_run:414: local output
>>>> > stderr: + 17:14:12: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:12: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:417: local var
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:426: echo -err
>>>> > stderr: ++ 17:14:12: ocf_run:426: sed -e s/-//g
>>>> > stderr: + 17:14:12: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:427: shift 1
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; kill -s 0 6046 >/dev/null 2>&1'
>>>> > stderr: + 17:14:12: ocf_run:433: output=
>>>> > stderr: + 17:14:12: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:12: ocf_run:435: echo
>>>> > stderr: + 17:14:12: ocf_run:435: output=
>>>> > stderr: + 17:14:12: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:12: ocf_run:437: '[' 1 -a '!' -z '' ']'
>>>> > stderr: + 17:14:12: ocf_run:440: return 0
>>>> > stderr: + 17:14:12: pgsql_status:870: return 0
>>>> > stderr: + 17:14:12: pgsql_real_stop:772: stop_escalate=30
>>>> > stderr: + 17:14:12: pgsql_real_stop:773: '[' slave = slave ']'
>>>> > stderr: + 17:14:12: pgsql_real_stop:774: stop_escalate=30
>>>> > stderr: + 17:14:12: pgsql_real_stop:778: '[' 30 -gt 0 ']'
>>>> > stderr: + 17:14:12: pgsql_real_stop:779: runasowner
>>>> '/usr/pgsql-9.3/bin/pg_ctl -D /var/lib/pgsql/9.3/data/ stop -m fast'
>>>> > stderr: + 17:14:12: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:12: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:12: runasowner:469: local var
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:12: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:12: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; /usr/pgsql-9.3/bin/pg_ctl
>>>> -D /var/lib/pgsql/9.3/data/ stop -m fast'
>>>> > stderr: + 17:14:12: ocf_run:413: local rc
>>>> > stderr: + 17:14:12: ocf_run:414: local output
>>>> > stderr: + 17:14:12: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:12: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:417: local var
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:426: echo -err
>>>> > stderr: ++ 17:14:12: ocf_run:426: sed -e s/-//g
>>>> > stderr: + 17:14:12: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:12: ocf_run:427: shift 1
>>>> > stderr: + 17:14:12: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:12: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:12: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; /usr/pgsql-9.3/bin/pg_ctl -D
>>>> /var/lib/pgsql/9.3/data/ stop -m fast'
>>>> > stderr: + 17:14:13: ocf_run:433: output='waiting for server to
>>>> shut down.... done
>>>> > stderr: server stopped'
>>>> > stderr: + 17:14:13: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:13: ocf_run:435: echo waiting for server to
>>>> shut down.... done server stopped
>>>> > stderr: + 17:14:13: ocf_run:435: output='waiting for server to
>>>> shut down.... done server stopped'
>>>> > stderr: + 17:14:13: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:13: ocf_run:437: '[' 1 -a '!' -z 'waiting for
>>>> server to shut down.... done server stopped' ']'
>>>> > stderr: + 17:14:13: ocf_run:438: ocf_log info 'waiting for
>>>> server to shut down.... done server stopped'
>>>> > stderr: + 17:14:13: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:13: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:13: ocf_log:326: shift
>>>> > stderr: + 17:14:13: ocf_log:327: __OCF_MSG='waiting for server
>>>> to shut down.... done server stopped'
>>>> > stderr: + 17:14:13: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:13: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:13: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:13: ocf_log:341: ha_log 'INFO: waiting for
>>>> server to shut down.... done server stopped'
>>>> > stderr: + 17:14:13: ha_log:253: __ha_log 'INFO: waiting for
>>>> server to shut down.... done server stopped'
>>>> > stderr: + 17:14:13: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:13: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:13: __ha_log:188: '[' 'xINFO: waiting for
>>>> server to shut down.... done server stopped' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:13: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:13: __ha_log:192: tty
>>>> > stderr: + 17:14:13: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:13: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:13: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:13: __ha_log:202: echo 'INFO: waiting for
>>>> server to shut down.... done server stopped'
>>>> > stderr: INFO: waiting for server to shut down.... done server
>>>> stopped
>>>> > stderr: + 17:14:13: __ha_log:204: return 0
>>>> > stderr: + 17:14:13: ocf_run:440: return 0
>>>> > stderr: + 17:14:13: pgsql_real_stop:783: count=0
>>>> > stderr: + 17:14:13: pgsql_real_stop:784: '[' 0 -lt 30 ']'
>>>> > stderr: + 17:14:13: pgsql_real_stop:786: pgsql_status
>>>> > stderr: + 17:14:13: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: + 17:14:13: pgsql_status:874: false
>>>> > stderr: + 17:14:13: pgsql_real_stop:789: break
>>>> > stderr: + 17:14:13: pgsql_real_stop:795: pgsql_status
>>>> > stderr: + 17:14:13: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: + 17:14:13: pgsql_status:874: false
>>>> > stderr: + 17:14:13: pgsql_real_stop:802: :
>>>> > stderr: + 17:14:13: pgsql_real_stop:804: pgsql_real_monitor
>>>> > stderr: + 17:14:13: pgsql_real_monitor:905: local loglevel
>>>> > stderr: + 17:14:13: pgsql_real_monitor:906: local rc
>>>> > stderr: + 17:14:13: pgsql_real_monitor:907: local output
>>>> > stderr: + 17:14:13: pgsql_real_monitor:910: loglevel=err
>>>> > stderr: + 17:14:13: pgsql_real_monitor:912: pgsql_status
>>>> > stderr: + 17:14:13: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: + 17:14:13: pgsql_status:874: false
>>>> > stderr: + 17:14:13: pgsql_real_monitor:914: ocf_log info
>>>> 'PostgreSQL is down'
>>>> > stderr: + 17:14:13: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:13: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:13: ocf_log:326: shift
>>>> > stderr: + 17:14:13: ocf_log:327: __OCF_MSG='PostgreSQL is down'
>>>> > stderr: + 17:14:13: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:13: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:13: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:13: ocf_log:341: ha_log 'INFO: PostgreSQL is down'
>>>> > stderr: + 17:14:13: ha_log:253: __ha_log 'INFO: PostgreSQL is
>>>> down'
>>>> > stderr: + 17:14:13: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:13: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:13: __ha_log:188: '[' 'xINFO: PostgreSQL is
>>>> down' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:13: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:13: __ha_log:192: tty
>>>> > stderr: + 17:14:13: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:13: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:13: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:13: __ha_log:202: echo 'INFO: PostgreSQL is down'
>>>> > stderr: INFO: PostgreSQL is down
>>>> > stderr: + 17:14:13: __ha_log:204: return 0
>>>> > stderr: + 17:14:13: pgsql_real_monitor:915: return 7
>>>> > stderr: + 17:14:13: pgsql_real_stop:805: rc=7
>>>> > stderr: + 17:14:13: pgsql_real_stop:806: '[' 7 -eq 7 ']'
>>>> > stderr: + 17:14:13: pgsql_real_stop:808: break
>>>> > stderr: + 17:14:13: pgsql_real_stop:815: rm -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid
>>>> > stderr: + 17:14:13: pgsql_real_stop:817: '[' slave = master -a
>>>> '' = ' ' ']'
>>>> > stderr: + 17:14:13: pgsql_real_stop:821: return 0
>>>> > stderr: + 17:14:13: pgsql_promote:688: rm -f
>>>> /var/lib/pgsql/9.3/data//recovery.conf
>>>> > stderr: + 17:14:13: pgsql_promote:689: pgsql_real_start
>>>> > stderr: + 17:14:13: pgsql_real_start:534: local pgctl_options
>>>> > stderr: + 17:14:13: pgsql_real_start:535: local postgres_options
>>>> > stderr: + 17:14:13: pgsql_real_start:536: local rc
>>>> > stderr: + 17:14:13: pgsql_real_start:538: pgsql_status
>>>> > stderr: + 17:14:13: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: + 17:14:13: pgsql_status:874: false
>>>> > stderr: + 17:14:13: pgsql_real_start:548: rm -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid
>>>> > stderr: + 17:14:13: pgsql_real_start:551: '[' -f
>>>> /var/lib/pgsql/9.3/data//backup_label ']'
>>>> > stderr: + 17:14:13: pgsql_real_start:557: check_log_file /dev/null
>>>> > stderr: + 17:14:13: check_log_file:1878: '[' '!' -f /dev/null ']'
>>>> > stderr: + 17:14:13: check_log_file:1880: touch /dev/null
>>>> > stderr: ++ 17:14:13: check_log_file:1881: getent passwd postgres
>>>> > stderr: ++ 17:14:13: check_log_file:1881: cut -d : -f 4
>>>> > stderr: + 17:14:13: check_log_file:1881: chown postgres:26
>>>> /dev/null
>>>> > stderr: + 17:14:13: check_log_file:1885: runasowner 'test -w
>>>> /dev/null'
>>>> > stderr: + 17:14:13: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:13: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:13: runasowner:469: local var
>>>> > stderr: + 17:14:13: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:13: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:13: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:13: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:13: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; test -w /dev/null'
>>>> > stderr: + 17:14:13: ocf_run:413: local rc
>>>> > stderr: + 17:14:13: ocf_run:414: local output
>>>> > stderr: + 17:14:13: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:13: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:13: ocf_run:417: local var
>>>> > stderr: + 17:14:13: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:13: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:13: ocf_run:426: echo -err
>>>> > stderr: ++ 17:14:13: ocf_run:426: sed -e s/-//g
>>>> > stderr: + 17:14:13: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:13: ocf_run:427: shift 1
>>>> > stderr: + 17:14:13: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:13: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:13: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; test -w /dev/null'
>>>> > stderr: + 17:14:13: ocf_run:433: output=
>>>> > stderr: + 17:14:13: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:13: ocf_run:435: echo
>>>> > stderr: + 17:14:13: ocf_run:435: output=
>>>> > stderr: + 17:14:13: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:13: ocf_run:437: '[' 1 -a '!' -z '' ']'
>>>> > stderr: + 17:14:13: ocf_run:440: return 0
>>>> > stderr: + 17:14:13: check_log_file:1890: return 0
>>>> > stderr: + 17:14:13: pgsql_real_start:564: '[' -n '' ']'
>>>> > stderr: + 17:14:13: pgsql_real_start:569: '[' sync = slave ']'
>>>> > stderr: + 17:14:13: pgsql_real_start:575: pgctl_options=' -D
>>>> /var/lib/pgsql/9.3/data/ -l /dev/null'
>>>> > stderr: + 17:14:13: pgsql_real_start:578: postgres_options='-c
>>>> config_file=/var/lib/pgsql/9.3/data//postgresql.conf'
>>>> > stderr: + 17:14:13: pgsql_real_start:580: '[' -n '' ']'
>>>> > stderr: + 17:14:13: pgsql_real_start:583: '[' -n '' ']'
>>>> > stderr: + 17:14:13: pgsql_real_start:588: pgctl_options=' -D
>>>> /var/lib/pgsql/9.3/data/ -l /dev/null -o '\''-c
>>>> config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\'''
>>>> > stderr: + 17:14:13: pgsql_real_start:591: runasowner 'unset
>>>> PGUSER; unset PGPASSWORD; /usr/pgsql-9.3/bin/pg_ctl -D
>>>> /var/lib/pgsql/9.3/data/ -l /dev/null -o '\''-c
>>>> config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\'' start'
>>>> > stderr: + 17:14:13: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:13: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:13: runasowner:469: local var
>>>> > stderr: + 17:14:13: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:13: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:13: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:13: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:13: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; unset PGUSER; unset
>>>> PGPASSWORD; /usr/pgsql-9.3/bin/pg_ctl -D /var/lib/pgsql/9.3/data/
>>>> -l /dev/null -o '\''-c
>>>> config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\'' start'
>>>> > stderr: + 17:14:13: ocf_run:413: local rc
>>>> > stderr: + 17:14:13: ocf_run:414: local output
>>>> > stderr: + 17:14:13: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:13: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:13: ocf_run:417: local var
>>>> > stderr: + 17:14:13: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:13: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:13: ocf_run:426: sed -e s/-//g
>>>> > stderr: ++ 17:14:13: ocf_run:426: echo -err
>>>> > stderr: + 17:14:13: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:13: ocf_run:427: shift 1
>>>> > stderr: + 17:14:13: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:13: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:13: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; unset PGUSER; unset PGPASSWORD;
>>>> /usr/pgsql-9.3/bin/pg_ctl -D /var/lib/pgsql/9.3/data/ -l /dev/null
>>>> -o '\''-c config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\''
>>>> start'
>>>> > stderr: + 17:14:13: ocf_run:433: output='server starting'
>>>> > stderr: + 17:14:13: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:13: ocf_run:435: echo server starting
>>>> > stderr: + 17:14:13: ocf_run:435: output='server starting'
>>>> > stderr: + 17:14:13: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:13: ocf_run:437: '[' 1 -a '!' -z 'server
>>>> starting' ']'
>>>> > stderr: + 17:14:13: ocf_run:438: ocf_log info 'server starting'
>>>> > stderr: + 17:14:13: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:13: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:13: ocf_log:326: shift
>>>> > stderr: + 17:14:13: ocf_log:327: __OCF_MSG='server starting'
>>>> > stderr: + 17:14:13: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:13: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: server starting'
>>>> > stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: server starting'
>>>> > stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:14: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: server starting'
>>>> = x--ignore-stderr ']'
>>>> > stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:14: __ha_log:192: tty
>>>> > stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:14: __ha_log:202: echo 'INFO: server starting'
>>>> > stderr: INFO: server starting
>>>> > stderr: + 17:14:14: __ha_log:204: return 0
>>>> > stderr: + 17:14:14: ocf_run:440: return 0
>>>> > stderr: + 17:14:14: pgsql_real_start:593: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:14: pgsql_real_start:595: ocf_log info
>>>> 'PostgreSQL start command sent.'
>>>> > stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:14: ocf_log:326: shift
>>>> > stderr: + 17:14:14: ocf_log:327: __OCF_MSG='PostgreSQL start
>>>> command sent.'
>>>> > stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:14: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: PostgreSQL start
>>>> command sent.'
>>>> > stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: PostgreSQL
>>>> start command sent.'
>>>> > stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:14: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: PostgreSQL start
>>>> command sent.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:14: __ha_log:192: tty
>>>> > stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:14: __ha_log:202: echo 'INFO: PostgreSQL start
>>>> command sent.'
>>>> > stderr: INFO: PostgreSQL start command sent.
>>>> > stderr: + 17:14:14: __ha_log:204: return 0
>>>> > stderr: + 17:14:14: pgsql_real_start:601: :
>>>> > stderr: + 17:14:14: pgsql_real_start:603: pgsql_real_monitor warn
>>>> > stderr: + 17:14:14: pgsql_real_monitor:905: local loglevel
>>>> > stderr: + 17:14:14: pgsql_real_monitor:906: local rc
>>>> > stderr: + 17:14:14: pgsql_real_monitor:907: local output
>>>> > stderr: + 17:14:14: pgsql_real_monitor:910: loglevel=warn
>>>> > stderr: + 17:14:14: pgsql_real_monitor:912: pgsql_status
>>>> > stderr: + 17:14:14: pgsql_status:866: '[' -f
>>>> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>>>> > stderr: ++ 17:14:14: pgsql_status:868: head -n 1
>>>> /var/lib/pgsql/9.3/data//postmaster.pid
>>>> > stderr: + 17:14:14: pgsql_status:868: PID=9556
>>>> > stderr: + 17:14:14: pgsql_status:869: runasowner 'kill -s 0
>>>> 9556 >/dev/null 2>&1'
>>>> > stderr: + 17:14:14: runasowner:467: local quietrun=
>>>> > stderr: + 17:14:14: runasowner:468: local loglevel=-err
>>>> > stderr: + 17:14:14: runasowner:469: local var
>>>> > stderr: + 17:14:14: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:14: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:14: runasowner:471: for var in 1 2
>>>> > stderr: + 17:14:14: runasowner:473: case "$1" in
>>>> > stderr: + 17:14:14: runasowner:485: ocf_run -err runuser
>>>> postgres -c 'cd /var/lib/pgsql/9.3/data/; kill -s 0 9556 >/dev/null
>>>> 2>&1'
>>>> > stderr: + 17:14:14: ocf_run:413: local rc
>>>> > stderr: + 17:14:14: ocf_run:414: local output
>>>> > stderr: + 17:14:14: ocf_run:415: local verbose=1
>>>> > stderr: + 17:14:14: ocf_run:416: local loglevel=err
>>>> > stderr: + 17:14:14: ocf_run:417: local var
>>>> > stderr: + 17:14:14: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:14: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:14: ocf_run:426: echo -err
>>>> > stderr: ++ 17:14:14: ocf_run:426: sed -e s/-//g
>>>> > stderr: + 17:14:14: ocf_run:426: loglevel=err
>>>> > stderr: + 17:14:14: ocf_run:427: shift 1
>>>> > stderr: + 17:14:14: ocf_run:419: for var in 1 2
>>>> > stderr: + 17:14:14: ocf_run:421: case "$1" in
>>>> > stderr: ++ 17:14:14: ocf_run:433: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; kill -s 0 9556 >/dev/null 2>&1'
>>>> > stderr: + 17:14:14: ocf_run:433: output=
>>>> > stderr: + 17:14:14: ocf_run:434: rc=0
>>>> > stderr: ++ 17:14:14: ocf_run:435: echo
>>>> > stderr: + 17:14:14: ocf_run:435: output=
>>>> > stderr: + 17:14:14: ocf_run:436: '[' 0 -eq 0 ']'
>>>> > stderr: + 17:14:14: ocf_run:437: '[' 1 -a '!' -z '' ']'
>>>> > stderr: + 17:14:14: ocf_run:440: return 0
>>>> > stderr: + 17:14:14: pgsql_status:870: return 0
>>>> > stderr: + 17:14:14: pgsql_real_monitor:918: is_replication
>>>> > stderr: + 17:14:14: is_replication:1287: '[' sync '!=' none -a
>>>> sync '!=' slave ']'
>>>> > stderr: + 17:14:14: is_replication:1288: return 0
>>>> > stderr: ++ 17:14:14: pgsql_real_monitor:920: exec_sql 'select
>>>> pg_is_in_recovery()'
>>>> > stderr: ++ 17:14:14: exec_sql:518: local 'sql=select
>>>> pg_is_in_recovery()'
>>>> > stderr: ++ 17:14:14: exec_sql:519: local output
>>>> > stderr: ++ 17:14:14: exec_sql:520: local rc
>>>> > stderr: +++ 17:14:14: exec_sql:524: runuser postgres -c 'cd
>>>> /var/lib/pgsql/9.3/data/; /usr/pgsql-9.3/bin/psql -p 5432 -U
>>>> postgres template1 -U postgres -Atc "select
>>>> pg_is_in_recovery()"'
>>>> > stderr: ++ 17:14:14: exec_sql:524: output=f
>>>> > stderr: ++ 17:14:14: exec_sql:525: rc=0
>>>> > stderr: ++ 17:14:14: exec_sql:527: echo f
>>>> > stderr: ++ 17:14:14: exec_sql:528: return 0
>>>> > stderr: + 17:14:14: pgsql_real_monitor:920: output=f
>>>> > stderr: + 17:14:14: pgsql_real_monitor:921: rc=0
>>>> > stderr: + 17:14:14: pgsql_real_monitor:923: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:14: pgsql_real_monitor:928: case "$output" in
>>>> > stderr: + 17:14:14: pgsql_real_monitor:929: ocf_log debug
>>>> 'PostgreSQL is running as a primary.'
>>>> > stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=debug
>>>> > stderr: + 17:14:14: ocf_log:326: shift
>>>> > stderr: + 17:14:14: ocf_log:327: __OCF_MSG='PostgreSQL is
>>>> running as a primary.'
>>>> > stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:14: ocf_log:334: __OCF_PRIO=DEBUG
>>>> > stderr: + 17:14:14: ocf_log:338: '[' DEBUG = DEBUG ']'
>>>> > stderr: + 17:14:14: ocf_log:339: ha_debug 'DEBUG: PostgreSQL is
>>>> running as a primary.'
>>>> > stderr: + 17:14:14: ha_debug:258: '[' x = x0 ']'
>>>> > stderr: + 17:14:14: ha_debug:261: tty
>>>> > stderr: + 17:14:14: ha_debug:262: '[' '' ']'
>>>> > stderr: + 17:14:14: ha_debug:265: echo 'DEBUG: PostgreSQL is
>>>> running as a primary.'
>>>> > stderr: DEBUG: PostgreSQL is running as a primary.
>>>> > stderr: + 17:14:14: ha_debug:267: return 0
>>>> > stderr: + 17:14:14: pgsql_real_monitor:930: '[' 'select now();'
>>>> = 'select now();' ']'
>>>> > stderr: + 17:14:14: pgsql_real_monitor:931: return 8
>>>> > stderr: + 17:14:14: pgsql_real_start:604: rc=8
>>>> > stderr: + 17:14:14: pgsql_real_start:605: '[' 8 -eq 0 -o 8 -eq
>>>> 8 ']'
>>>> > stderr: + 17:14:14: pgsql_real_start:606: break
>>>> > stderr: + 17:14:14: pgsql_real_start:614: use_replication_slot
>>>> > stderr: + 17:14:14: use_replication_slot:1294: '[' -n '' ']'
>>>> > stderr: + 17:14:14: use_replication_slot:1298: return 1
>>>> > stderr: + 17:14:14: pgsql_real_start:622: ocf_log info
>>>> 'PostgreSQL is started.'
>>>> > stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:14: ocf_log:326: shift
>>>> > stderr: + 17:14:14: ocf_log:327: __OCF_MSG='PostgreSQL is
>>>> started.'
>>>> > stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:14: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: PostgreSQL is
>>>> started.'
>>>> > stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: PostgreSQL is
>>>> started.'
>>>> > stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:14: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: PostgreSQL is
>>>> started.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:14: __ha_log:192: tty
>>>> > stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:14: __ha_log:202: echo 'INFO: PostgreSQL is
>>>> started.'
>>>> > stderr: INFO: PostgreSQL is started.
>>>> > stderr: + 17:14:14: __ha_log:204: return 0
>>>> > stderr: + 17:14:14: pgsql_real_start:623: return 8
>>>> > stderr: + 17:14:14: pgsql_promote:690: rc=8
>>>> > stderr: + 17:14:14: pgsql_promote:691: '[' 8 -ne 8 ']'
>>>> > stderr: + 17:14:14: pgsql_promote:723: change_data_status pm1
>>>> LATEST
>>>> > stderr: + 17:14:14: change_data_status:1569: local output
>>>> > stderr: + 17:14:14: change_data_status:1571: node_exist pm1
>>>> > stderr: + 17:14:14: node_exist:1720: print_crm_mon
>>>> > stderr: + 17:14:14: node_exist:1720: grep -q '^node pm1'
>>>> > stderr: + 17:14:14: node_exist:1720: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:14: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:14: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:14: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:14: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:14: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:14: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:14: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:14: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:14: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: + 17:14:14: change_data_status:1575: :
>>>> > stderr: ++ 17:14:14: change_data_status:1577:
>>>> /usr/sbin/crm_attribute -l forever -N pm1 -n pgsql-data-status -G -q
>>>> > stderr: + 17:14:14: change_data_status:1577: output=LATEST
>>>> > stderr: + 17:14:14: change_data_status:1578: '[' LATEST '!='
>>>> LATEST ']'
>>>> > stderr: + 17:14:14: change_data_status:1582: break
>>>> > stderr: + 17:14:14: change_data_status:1585: return 0
>>>> > stderr: + 17:14:14: pgsql_promote:724: exec_with_retry 0
>>>> /usr/sbin/crm_master -l reboot -v 1000
>>>> > stderr: + 17:14:14: exec_with_retry:1689: local count=86400
>>>> > stderr: + 17:14:14: exec_with_retry:1690: local output
>>>> > stderr: + 17:14:14: exec_with_retry:1691: local rc
>>>> > stderr: + 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:14: exec_with_retry:1696: shift
>>>> > stderr: + 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1699: /usr/sbin/crm_master
>>>> -l reboot -v 1000
>>>> > stderr: + 17:14:14: exec_with_retry:1699: output=
>>>> > stderr: + 17:14:14: exec_with_retry:1700: rc=0
>>>> > stderr: + 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:14: exec_with_retry:1706: printf ''
>>>> > stderr: + 17:14:14: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:14: pgsql_promote:725: change_pgsql_status pm1 PRI
>>>> > stderr: + 17:14:14: change_pgsql_status:1543: local output
>>>> > stderr: + 17:14:14: change_pgsql_status:1545: is_node_online pm1
>>>> > stderr: + 17:14:14: is_node_online:1716: print_crm_mon
>>>> > stderr: + 17:14:14: is_node_online:1716: tr '[A-Z]' '[a-z]'
>>>> > stderr: + 17:14:14: is_node_online:1716: grep -e '^node pm1 '
>>>> -e '^node pm1:'
>>>> > stderr: + 17:14:14: is_node_online:1716: grep -q -v offline
>>>> > stderr: + 17:14:14: print_crm_mon:1924: '[' -z '' ']'
>>>> > stderr: ++ 17:14:14: print_crm_mon:1925: exec_with_retry 0
>>>> crm_mon -n1
>>>> > stderr: ++ 17:14:14: exec_with_retry:1689: local count=86400
>>>> > stderr: ++ 17:14:14: exec_with_retry:1690: local output
>>>> > stderr: ++ 17:14:14: exec_with_retry:1691: local rc
>>>> > stderr: ++ 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1696: shift
>>>> > stderr: ++ 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: +++ 17:14:14: exec_with_retry:1699: crm_mon -n1
>>>> > stderr: ++ 17:14:14: exec_with_retry:1699: output='Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1700: rc=0
>>>> > stderr: ++ 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1706: printf 'Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12
>>>> 16:07:56 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:14: print_crm_mon:1925: CRM_MON_OUTPUT='Last
>>>> updated: Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave'
>>>> > stderr: + 17:14:14: print_crm_mon:1927: printf 'Last updated:
>>>> Mon Oct 12 17:14:14 2015 Last change: Mon Oct 12 16:07:56
>>>> 2015 by root via crm_attribute on pm1
>>>> > stderr: Stack: corosync
>>>> > stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition
>>>> with quorum
>>>> > stderr: 2 nodes and 2 resources configured
>>>> > stderr:
>>>> > stderr: Node pm1: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Started
>>>> > stderr: Node pm2: online
>>>> > stderr: pgsql (ocf::heartbeat:pgsql): Slave\n'
>>>> > stderr: ++ 17:14:14: change_pgsql_status:1549:
>>>> /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-status -G -q
>>>> > stderr: + 17:14:14: change_pgsql_status:1549: output=HS:alone
>>>> > stderr: + 17:14:14: change_pgsql_status:1550: '[' HS:alone '!='
>>>> PRI ']'
>>>> > stderr: + 17:14:14: change_pgsql_status:1554: '[' HS:alone =
>>>> STOP -o HS:alone = UNKNOWN ']'
>>>> > stderr: + 17:14:14: change_pgsql_status:1560: ocf_log info
>>>> 'Changing pgsql-status on pm1 : HS:alone->PRI.'
>>>> > stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>>>> > stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=info
>>>> > stderr: + 17:14:14: ocf_log:326: shift
>>>> > stderr: + 17:14:14: ocf_log:327: __OCF_MSG='Changing
>>>> pgsql-status on pm1 : HS:alone->PRI.'
>>>> > stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>>>> > stderr: + 17:14:14: ocf_log:333: __OCF_PRIO=INFO
>>>> > stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>>>> > stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: Changing
>>>> pgsql-status on pm1 : HS:alone->PRI.'
>>>> > stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: Changing
>>>> pgsql-status on pm1 : HS:alone->PRI.'
>>>> > stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>>>> > stderr: + 17:14:14: __ha_log:186: local loglevel
>>>> > stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: Changing
>>>> pgsql-status on pm1 : HS:alone->PRI.' = x--ignore-stderr ']'
>>>> > stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>>>> > stderr: + 17:14:14: __ha_log:192: tty
>>>> > stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>>>> > stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>>>> > stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>>>> > stderr: + 17:14:14: __ha_log:202: echo 'INFO: Changing
>>>> pgsql-status on pm1 : HS:alone->PRI.'
>>>> > stderr: INFO: Changing pgsql-status on pm1 : HS:alone->PRI.
>>>> > stderr: + 17:14:14: __ha_log:204: return 0
>>>> > stderr: + 17:14:14: change_pgsql_status:1561: exec_with_retry 0
>>>> /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-status -v PRI
>>>> > stderr: + 17:14:14: exec_with_retry:1689: local count=86400
>>>> > stderr: + 17:14:14: exec_with_retry:1690: local output
>>>> > stderr: + 17:14:14: exec_with_retry:1691: local rc
>>>> > stderr: + 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:14: exec_with_retry:1696: shift
>>>> > stderr: + 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>>>> > stderr: ++ 17:14:14: exec_with_retry:1699:
>>>> /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-status -v PRI
>>>> > stderr: + 17:14:14: exec_with_retry:1699: output=
>>>> > stderr: + 17:14:14: exec_with_retry:1700: rc=0
>>>> > stderr: + 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>>>> > stderr: + 17:14:14: exec_with_retry:1706: printf ''
>>>> > stderr: + 17:14:14: exec_with_retry:1707: return 0
>>>> > stderr: + 17:14:14: change_pgsql_status:1563: return 0
>>>> > stderr: + 17:14:14: pgsql_promote:726: return 0
>>>> > stderr: + 17:14:14: 2058: exit 0
>>>>
>>>>
>>>> Thanks in advance!
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list: Users at clusterlabs.org
>>>> http://clusterlabs.org/mailman/listinfo/users
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started:
>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://bugs.clusterlabs.org
>>>
>>
>>
>> _______________________________________________
>> Users mailing list: Users at clusterlabs.org
>> http://clusterlabs.org/mailman/listinfo/users
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>
> --
> Nikolay Popov
> n.popov at postgrespro.ru
> Postgres Professional:http://www.postgrespro.com
> The Russian Postgres Company
>
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20151012/804aa87e/attachment-0003.html>
More information about the Users
mailing list