[ClusterLabs] pgsql Master/Slave set has no promote (all nodes are slaves)

Nikolay Popov n.popov at postgrespro.ru
Mon Oct 12 14:36:23 UTC 2015


Hi, Andrey.

You added VIP?
That is a VIP (master_ip="10.200.200.233")?



12.10.2015 17:23, Andrey Rubik пишет:
> Hi all.
>
> I'm trying to configure pgsql Master/Slave set with 2 nodes.
>
> I'm get successful installation Master/Slave without pacemaker.
>
> Pacemaker version:
>
> pacemakerd --features
> Pacemaker 1.1.13 (Build: a14efad)
>  Supporting v3.0.10:  generated-manpages agent-manpages ascii-docs 
> publican-docs ncurses libqb-logging libqb-ipc upstart systemd nagios  
> corosync-native atomic-attrd acls
>
> That's my configuration:
>
> pcs cluster cib pgsql_cfg
>
> pcs -f pgsql_cfg property set no-quorum-policy="ignore"
> pcs -f pgsql_cfg property set stonith-enabled="false"
>
> pcs -f pgsql_cfg resource create pgsql pgsql \
>    pgctl="/usr/pgsql-9.3/bin/pg_ctl" \
>    psql="/usr/pgsql-9.3/bin/psql" \
>    pgdata="/var/lib/pgsql/9.3/data/" \
>    tmpdir="/var/lib/pgsql/9.3/tmp/" \
>    rep_mode="sync" \
>    node_list="pg1 pg2" \
>    restore_command="cp /var/lib/pgsql/9.3/backups/%f %p" \
>    primary_conninfo_opt="keepalives_idle=60 keepalives_interval=5 
> keepalives_count=5" \
>    master_ip="10.200.200.233" \
>    restart_on_promote='true' \
>    op start   timeout="60s" interval="0s"  on-fail="restart" \
>    op monitor timeout="60s" interval="4s" on-fail="restart" \
>    op monitor timeout="60s" interval="3s"  on-fail="restart" 
> role="Master" \
>    op promote timeout="60s" interval="0s"  on-fail="restart" \
>    op demote  timeout="60s" interval="0s"  on-fail="stop" \
>    op stop    timeout="60s" interval="0s"  on-fail="block" \
>    op notify  timeout="60s" interval="0s"
>
> pcs -f pgsql_cfg resource master msPostgresql pgsql \
>    master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 
> notify=true
>
> pcs cluster cib-push pgsql_cfg
>
>
>
> crm_mon:
>
> Stack: corosync
> Current DC: pm2 (version 1.1.13-a14efad) - partition with quorum
> 2 nodes and 2 resources configured
>
> Online: [ pm1 pm2 ]
>
> Full list of resources:
>
>  Master/Slave Set: msPostgresql [pgsql]
>      Slaves: [ pm1 pm2 ]
>
> Node Attributes:
> * Node pm1:
>     + master-pgsql                      : -INFINITY
>     + pgsql-data-status                 : LATEST
>     + pgsql-status                      : HS:alone
>     + pgsql-xlog-loc                    : 0000000000000000
> * Node pm2:
>     + master-pgsql                      : -INFINITY
>     + pgsql-data-status                 : LATEST
>     + pgsql-master-baseline             : 0000000000000000
>     + pgsql-status                      : HS:alone
>     + pgsql-xlog-loc                    : 0000000000000000
>
> Migration Summary:
> * Node pm2:
> * Node pm1:
>
>
>
> /var/log/pacemaker.log:
>
> WARNING: Can't get pg1 xlog location.
> WARNING: Can't get pg2 xlog location.
> WARNING: Can't get pg1 xlog location.
> WARNING: Can't get pg2 xlog location.
> .....
>
> Its very strange messages since i have pgsql-xlog-loc in crm_mon.
>
>
>
>
> If I try set master manually it fails and then stop pm1 node:
>
> crm_resource -N pm1 -r pgsql --force-promote
>
> Operation promote for pgsql:0 (ocf:heartbeat:pgsql) returned 0
>  >  stderr: INFO: Creating /var/lib/pgsql/9.3/tmp//PGSQL.lock.
>  >  stderr: INFO: My master baseline : 0000000000000000.
>  >  stderr: INFO: Restarting PostgreSQL instead of promote.
>  >  stderr: INFO: waiting for server to shut down.... done server stopped
>  >  stderr: INFO: PostgreSQL is down
>  >  stderr: INFO: server starting
>  >  stderr: INFO: PostgreSQL start command sent.
>  >  stderr: psql: could not connect to server: No such file or directory
>  >  stderr:     Is the server running locally and accepting
>  >  stderr:     connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
>  >  stderr: WARNING: Can't get PostgreSQL recovery status. rc=2
>  >  stderr: WARNING: Connection error (connection to the server went 
> bad and the session was not interactive) occurred while executing the 
> psql command.
>  >  stderr: DEBUG: PostgreSQL still hasn't started yet. Waiting...
>  >  stderr: DEBUG: PostgreSQL is running as a primary.
>  >  stderr: INFO: PostgreSQL is started.
>  >  stderr: INFO: Changing pgsql-status on pm1 : HS:alone->PRI.
>
> crm_mon:
>
> Stack: corosync
> Current DC: pm2 (version 1.1.13-a14efad) - partition with quorum
> 2 nodes and 2 resources configured
>
> Online: [ pm1 pm2 ]
>
> Full list of resources:
>
>  Master/Slave Set: msPostgresql [pgsql]
>      Slaves: [ pm2 ]
>      Stopped: [ pm1 ]
>
> Node Attributes:
> * Node pm1:
>     + master-pgsql                      : -INFINITY
>     + master-pgsql:0                    : 1000
>     + pgsql-data-status                 : LATEST
>     + pgsql-status                      : STOP
> * Node pm2:
>     + master-pgsql                      : -INFINITY
>     + pgsql-data-status                 : LATEST
>     + pgsql-master-baseline             : 0000000000000000
>     + pgsql-status                      : HS:alone
>     + pgsql-xlog-loc                    : 0000000000000000
>
> Migration Summary:
> * Node pm2:
> * Node pm1:
>    pgsql: migration-threshold=1000000 fail-count=1000000 
> last-failure='Mon Oct 12 17:12:04 2015'
>
> Failed Actions:
> * pgsql_start_0 on pm1 'unknown error' (1): call=47, status=complete, 
> exitreason='My data may be inconsistent. You have to remove 
> /var/lib/pgsql/9.3/tmp//PGSQL.lock file to force start.',
>     last-rc-change='Mon Oct 12 17:12:04 2015', queued=0ms, exec=170ms
>
>
>
> That is verbose variant:
> rm -f /var/lib/pgsql/9.3/tmp//PGSQL.lock
> crm_resource -N pm1 -r pgsql --force-promote -V
> Operation promote for pgsql:0 (ocf:heartbeat:pgsql) returned 0
>  >  stderr: + 17:14:11: 22: '[' -x /sbin/runuser ']'
>  >  stderr: + 17:14:11: 23: SU=runuser
>  >  stderr: + 17:14:11: 45: OCF_RESKEY_pgctl_default=/usr/bin/pg_ctl
>  >  stderr: + 17:14:11: 46: OCF_RESKEY_psql_default=/usr/bin/psql
>  >  stderr: + 17:14:11: 47: OCF_RESKEY_pgdata_default=/var/lib/pgsql/data
>  >  stderr: + 17:14:11: 48: OCF_RESKEY_pgdba_default=postgres
>  >  stderr: + 17:14:11: 49: OCF_RESKEY_pghost_default=
>  >  stderr: + 17:14:11: 50: OCF_RESKEY_pgport_default=5432
>  >  stderr: + 17:14:11: 51: OCF_RESKEY_pglibs_default=/usr/lib
>  >  stderr: + 17:14:11: 52: OCF_RESKEY_start_opt_default=
>  >  stderr: + 17:14:11: 53: OCF_RESKEY_pgdb_default=template1
>  >  stderr: + 17:14:11: 54: OCF_RESKEY_logfile_default=/dev/null
>  >  stderr: + 17:14:11: 55: OCF_RESKEY_stop_escalate_default=30
>  >  stderr: + 17:14:11: 56: OCF_RESKEY_monitor_user_default=
>  >  stderr: + 17:14:11: 57: OCF_RESKEY_monitor_password_default=
>  >  stderr: + 17:14:11: 58: OCF_RESKEY_monitor_sql_default='select 
> now();'
>  >  stderr: + 17:14:11: 59: OCF_RESKEY_check_wal_receiver_default=false
>  >  stderr: + 17:14:11: 61: OCF_RESKEY_rep_mode_default=none
>  >  stderr: + 17:14:11: 62: OCF_RESKEY_node_list_default=
>  >  stderr: + 17:14:11: 63: OCF_RESKEY_restore_command_default=
>  >  stderr: + 17:14:11: 64: OCF_RESKEY_archive_cleanup_command_default=
>  >  stderr: + 17:14:11: 65: OCF_RESKEY_recovery_end_command_default=
>  >  stderr: + 17:14:11: 66: OCF_RESKEY_master_ip_default=
>  >  stderr: + 17:14:11: 67: OCF_RESKEY_repuser_default=postgres
>  >  stderr: + 17:14:11: 68: OCF_RESKEY_primary_conninfo_opt_default=
>  >  stderr: + 17:14:11: 69: OCF_RESKEY_restart_on_promote_default=false
>  >  stderr: + 17:14:11: 70: OCF_RESKEY_tmpdir_default=/var/lib/pgsql/tmp
>  >  stderr: + 17:14:11: 71: OCF_RESKEY_xlog_check_count_default=3
>  >  stderr: + 17:14:11: 72: OCF_RESKEY_crm_attr_timeout_default=5
>  >  stderr: + 17:14:11: 73: OCF_RESKEY_stop_escalate_in_slave_default=30
>  >  stderr: + 17:14:11: 74: OCF_RESKEY_replication_slot_name_default=
>  >  stderr: + 17:14:11: 76: : /usr/pgsql-9.3/bin/pg_ctl
>  >  stderr: + 17:14:11: 77: : /usr/pgsql-9.3/bin/psql
>  >  stderr: + 17:14:11: 78: : /var/lib/pgsql/9.3/data/
>  >  stderr: + 17:14:11: 79: : postgres
>  >  stderr: + 17:14:11: 80: :
>  >  stderr: + 17:14:11: 81: : 5432
>  >  stderr: + 17:14:11: 82: : /usr/lib
>  >  stderr: + 17:14:11: 83: : /var/lib/pgsql/9.3/data//postgresql.conf
>  >  stderr: + 17:14:11: 84: :
>  >  stderr: + 17:14:11: 85: : template1
>  >  stderr: + 17:14:11: 86: : /dev/null
>  >  stderr: + 17:14:11: 87: : 30
>  >  stderr: + 17:14:11: 88: :
>  >  stderr: + 17:14:11: 89: :
>  >  stderr: + 17:14:11: 90: : select 'now();'
>  >  stderr: + 17:14:11: 91: : false
>  >  stderr: + 17:14:11: 94: : sync
>  >  stderr: + 17:14:11: 95: : pg1 pg2
>  >  stderr: + 17:14:11: 96: : cp /var/lib/pgsql/9.3/backups/%f %p
>  >  stderr: + 17:14:11: 97: :
>  >  stderr: + 17:14:11: 98: :
>  >  stderr: + 17:14:11: 99: : 10.200.200.233
>  >  stderr: + 17:14:11: 100: : postgres
>  >  stderr: + 17:14:11: 101: : keepalives_idle=60 
> keepalives_interval=5 keepalives_count=5
>  >  stderr: + 17:14:11: 102: : true
>  >  stderr: + 17:14:11: 103: : /var/lib/pgsql/9.3/tmp/
>  >  stderr: + 17:14:11: 104: : 3
>  >  stderr: + 17:14:11: 105: : 5
>  >  stderr: + 17:14:11: 106: : 30
>  >  stderr: + 17:14:11: 107: :
>  >  stderr: + 17:14:11: 1935: '[' 1 -ne 1 ']'
>  >  stderr: + 17:14:11: 1942: 
> PIDFILE=/var/lib/pgsql/9.3/data//postmaster.pid
>  >  stderr: + 17:14:11: 1943: 
> BACKUPLABEL=/var/lib/pgsql/9.3/data//backup_label
>  >  stderr: ++ 17:14:11: 1944: echo pgsql:0
>  >  stderr: ++ 17:14:11: 1944: cut -d : -f 1
>  >  stderr: + 17:14:11: 1944: RESOURCE_NAME=pgsql
>  >  stderr: + 17:14:11: 1945: 
> PGSQL_WAL_RECEIVER_STATUS_ATTR=pgsql-receiver-status
>  >  stderr: + 17:14:11: 1946: 
> RECOVERY_CONF=/var/lib/pgsql/9.3/data//recovery.conf
>  >  stderr: ++ 17:14:11: 1947: ocf_local_nodename
>  >  stderr: ++ 17:14:11: 1947: tr '[A-Z]' '[a-z]'
>  >  stderr: ++ 17:14:11: ocf_local_nodename:570: which pacemakerd
>  >  stderr: ++ 17:14:11: ocf_local_nodename:571: '[' 0 -eq 0 ']'
>  >  stderr: +++ 17:14:11: ocf_local_nodename:572: pacemakerd '-$'
>  >  stderr: +++ 17:14:12: ocf_local_nodename:572: grep 'Pacemaker .*'
>  >  stderr: +++ 17:14:12: ocf_local_nodename:572: awk '{ print $2 }'
>  >  stderr: ++ 17:14:12: ocf_local_nodename:572: local version=1.1.13
>  >  stderr: +++ 17:14:12: ocf_local_nodename:573: echo 1.1.13
>  >  stderr: +++ 17:14:12: ocf_local_nodename:573: awk -F- '{ print $1 }'
>  >  stderr: ++ 17:14:12: ocf_local_nodename:573: version=1.1.13
>  >  stderr: ++ 17:14:12: ocf_local_nodename:574: ocf_version_cmp 
> 1.1.13 1.1.8
>  >  stderr: ++ 17:14:12: ocf_version_cmp:543: ocf_is_ver 1.1.13
>  >  stderr: ++ 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>  >  stderr: ++ 17:14:12: ocf_is_ver:513: echo 1.1.13
>  >  stderr: ++ 17:14:12: ocf_version_cmp:544: ocf_is_ver 1.1.8
>  >  stderr: ++ 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>  >  stderr: ++ 17:14:12: ocf_is_ver:513: echo 1.1.8
>  >  stderr: ++ 17:14:12: ocf_version_cmp:545: local v1=1.1.13
>  >  stderr: ++ 17:14:12: ocf_version_cmp:546: local v2=1.1.8
>  >  stderr: +++ 17:14:12: ocf_version_cmp:547: ocf_ver_level 1.1.13
>  >  stderr: +++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>  >  stderr: +++ 17:14:12: ocf_ver_level:521: echo 1.1.13
>  >  stderr: ++ 17:14:12: ocf_version_cmp:547: local v1_level=3
>  >  stderr: +++ 17:14:12: ocf_version_cmp:548: ocf_ver_level 1.1.8
>  >  stderr: +++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>  >  stderr: +++ 17:14:12: ocf_ver_level:521: echo 1.1.8
>  >  stderr: ++ 17:14:12: ocf_version_cmp:548: local v2_level=3
>  >  stderr: ++ 17:14:12: ocf_version_cmp:549: local level_diff
>  >  stderr: ++ 17:14:12: ocf_version_cmp:550: '[' 3 -lt 3 ']'
>  >  stderr: ++ 17:14:12: ocf_version_cmp:553: '[' 3 -gt 3 ']'
>  >  stderr: +++ 17:14:12: ocf_version_cmp:557: ocf_ver2num 1.1.13
>  >  stderr: +++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>  >  stderr:     {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>  >  stderr:     '
>  >  stderr: +++ 17:14:12: ocf_ver2num:516: echo 1.1.13
>  >  stderr: ++ 17:14:12: ocf_version_cmp:557: v1=1001013
>  >  stderr: +++ 17:14:12: ocf_version_cmp:558: ocf_ver2num 1.1.8
>  >  stderr: +++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>  >  stderr:     {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>  >  stderr:     '
>  >  stderr: +++ 17:14:12: ocf_ver2num:516: echo 1.1.8
>  >  stderr: ++ 17:14:12: ocf_version_cmp:558: v2=1001008
>  >  stderr: ++ 17:14:12: ocf_version_cmp:559: '[' 1001013 -eq 1001008 ']'
>  >  stderr: ++ 17:14:12: ocf_version_cmp:561: '[' 1001013 -lt 1001008 ']'
>  >  stderr: ++ 17:14:12: ocf_version_cmp:564: return 2
>  >  stderr: ++ 17:14:12: ocf_local_nodename:575: '[' 2 -eq 2 ']'
>  >  stderr: ++ 17:14:12: ocf_local_nodename:576: which crm_node
>  >  stderr: ++ 17:14:12: ocf_local_nodename:577: '[' 0 -eq 0 ']'
>  >  stderr: ++ 17:14:12: ocf_local_nodename:578: crm_node -n
>  >  stderr: ++ 17:14:12: ocf_local_nodename:579: return
>  >  stderr: + 17:14:12: 1947: NODENAME=pm1
>  >  stderr: + 17:14:12: 1949: is_replication
>  >  stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a 
> sync '!=' slave ']'
>  >  stderr: + 17:14:12: is_replication:1288: return 0
>  >  stderr: + 17:14:12: 1950: 
> REP_MODE_CONF=/var/lib/pgsql/9.3/tmp//rep_mode.conf
>  >  stderr: + 17:14:12: 1951: 
> PGSQL_LOCK=/var/lib/pgsql/9.3/tmp//PGSQL.lock
>  >  stderr: + 17:14:12: 1952: 
> XLOG_NOTE_FILE=/var/lib/pgsql/9.3/tmp//xlog_note
>  >  stderr: + 17:14:12: 1954: CRM_MASTER='/usr/sbin/crm_master -l reboot'
>  >  stderr: + 17:14:12: 1955: CRM_ATTR_REBOOT='/usr/sbin/crm_attribute 
> -l reboot'
>  >  stderr: + 17:14:12: 1956: 
> CRM_ATTR_FOREVER='/usr/sbin/crm_attribute -l forever'
>  >  stderr: + 17:14:12: 1957: CRM_FAILCOUNT=/usr/sbin/crm_failcount
>  >  stderr: + 17:14:12: 1959: CAN_NOT_PROMOTE=-INFINITY
>  >  stderr: + 17:14:12: 1960: CAN_PROMOTE=100
>  >  stderr: + 17:14:12: 1961: PROMOTE_ME=1000
>  >  stderr: + 17:14:12: 1963: CHECK_MS_SQL='select pg_is_in_recovery()'
>  >  stderr: + 17:14:12: 1964: CHECK_XLOG_LOC_SQL='select 
> pg_last_xlog_replay_location(),pg_last_xlog_receive_location()'
>  >  stderr: + 17:14:12: 1965: CHECK_REPLICATION_STATE_SQL='select 
> application_name,upper(state),upper(sync_state) from pg_stat_replication'
>  >  stderr: + 17:14:12: 1967: PGSQL_STATUS_ATTR=pgsql-status
>  >  stderr: + 17:14:12: 1968: PGSQL_DATA_STATUS_ATTR=pgsql-data-status
>  >  stderr: + 17:14:12: 1969: PGSQL_XLOG_LOC_NAME=pgsql-xlog-loc
>  >  stderr: + 17:14:12: 1970: PGSQL_MASTER_BASELINE=pgsql-master-baseline
>  >  stderr: ++ 17:14:12: 1972: echo pg1 pg2
>  >  stderr: ++ 17:14:12: 1972: tr '[A-Z]' '[a-z]'
>  >  stderr: + 17:14:12: 1972: NODE_LIST='pg1 pg2'
>  >  stderr: + 17:14:12: 1973: RE_CONTROL_SLAVE=false
>  >  stderr: + 17:14:12: 1976: case "$1" in
>  >  stderr: + 17:14:12: 1984: pgsql_validate_all
>  >  stderr: + 17:14:12: pgsql_validate_all:1749: local version
>  >  stderr: + 17:14:12: pgsql_validate_all:1750: local check_config_rc
>  >  stderr: + 17:14:12: pgsql_validate_all:1751: local rep_mode_string
>  >  stderr: + 17:14:12: pgsql_validate_all:1752: local socket_directories
>  >  stderr: ++ 17:14:12: pgsql_validate_all:1754: cat 
> /var/lib/pgsql/9.3/data//PG_VERSION
>  >  stderr: + 17:14:12: pgsql_validate_all:1754: version=9.3
>  >  stderr: + 17:14:12: pgsql_validate_all:1756: check_binary2 
> /usr/pgsql-9.3/bin/pg_ctl
>  >  stderr: + 17:14:12: check_binary2:1724: have_binary 
> /usr/pgsql-9.3/bin/pg_ctl
>  >  stderr: + 17:14:12: have_binary:68: '[' '' = 1 ']'
>  >  stderr: ++ 17:14:12: have_binary:71: echo /usr/pgsql-9.3/bin/pg_ctl
>  >  stderr: ++ 17:14:12: have_binary:71: sed -e 's/ -.*//'
>  >  stderr: + 17:14:12: have_binary:71: local 
> bin=/usr/pgsql-9.3/bin/pg_ctl
>  >  stderr: ++ 17:14:12: have_binary:72: which /usr/pgsql-9.3/bin/pg_ctl
>  >  stderr: + 17:14:12: have_binary:72: test -x /usr/pgsql-9.3/bin/pg_ctl
>  >  stderr: + 17:14:12: check_binary2:1728: return 0
>  >  stderr: + 17:14:12: pgsql_validate_all:1757: check_binary2 
> /usr/pgsql-9.3/bin/psql
>  >  stderr: + 17:14:12: check_binary2:1724: have_binary 
> /usr/pgsql-9.3/bin/psql
>  >  stderr: + 17:14:12: have_binary:68: '[' '' = 1 ']'
>  >  stderr: ++ 17:14:12: have_binary:71: echo /usr/pgsql-9.3/bin/psql
>  >  stderr: ++ 17:14:12: have_binary:71: sed -e 's/ -.*//'
>  >  stderr: + 17:14:12: have_binary:71: local bin=/usr/pgsql-9.3/bin/psql
>  >  stderr: ++ 17:14:12: have_binary:72: which /usr/pgsql-9.3/bin/psql
>  >  stderr: + 17:14:12: have_binary:72: test -x /usr/pgsql-9.3/bin/psql
>  >  stderr: + 17:14:12: check_binary2:1728: return 0
>  >  stderr: + 17:14:12: pgsql_validate_all:1761: check_config 
> /var/lib/pgsql/9.3/data//postgresql.conf
>  >  stderr: + 17:14:12: check_config:1732: local rc=0
>  >  stderr: + 17:14:12: check_config:1734: '[' '!' -f 
> /var/lib/pgsql/9.3/data//postgresql.conf ']'
>  >  stderr: + 17:14:12: check_config:1744: return 0
>  >  stderr: + 17:14:12: pgsql_validate_all:1762: check_config_rc=0
>  >  stderr: + 17:14:12: pgsql_validate_all:1763: '[' 0 -eq 2 ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1764: '[' 0 -eq 0 ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1765: ocf_version_cmp 9.3 9.3
>  >  stderr: + 17:14:12: ocf_version_cmp:543: ocf_is_ver 9.3
>  >  stderr: + 17:14:12: ocf_is_ver:513: echo 9.3
>  >  stderr: + 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>  >  stderr: + 17:14:12: ocf_version_cmp:544: ocf_is_ver 9.3
>  >  stderr: + 17:14:12: ocf_is_ver:513: grep '^[0-9][0-9.-]*[0-9]$'
>  >  stderr: + 17:14:12: ocf_is_ver:513: echo 9.3
>  >  stderr: + 17:14:12: ocf_version_cmp:545: local v1=9.3
>  >  stderr: + 17:14:12: ocf_version_cmp:546: local v2=9.3
>  >  stderr: ++ 17:14:12: ocf_version_cmp:547: ocf_ver_level 9.3
>  >  stderr: ++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>  >  stderr: ++ 17:14:12: ocf_ver_level:521: echo 9.3
>  >  stderr: + 17:14:12: ocf_version_cmp:547: local v1_level=2
>  >  stderr: ++ 17:14:12: ocf_version_cmp:548: ocf_ver_level 9.3
>  >  stderr: ++ 17:14:12: ocf_ver_level:521: awk '-F[.-]' '{print NF}'
>  >  stderr: ++ 17:14:12: ocf_ver_level:521: echo 9.3
>  >  stderr: + 17:14:12: ocf_version_cmp:548: local v2_level=2
>  >  stderr: + 17:14:12: ocf_version_cmp:549: local level_diff
>  >  stderr: + 17:14:12: ocf_version_cmp:550: '[' 2 -lt 2 ']'
>  >  stderr: + 17:14:12: ocf_version_cmp:553: '[' 2 -gt 2 ']'
>  >  stderr: ++ 17:14:12: ocf_version_cmp:557: ocf_ver2num 9.3
>  >  stderr: ++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>  >  stderr:     {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>  >  stderr:     '
>  >  stderr: ++ 17:14:12: ocf_ver2num:516: echo 9.3
>  >  stderr: + 17:14:12: ocf_version_cmp:557: v1=9003
>  >  stderr: ++ 17:14:12: ocf_version_cmp:558: ocf_ver2num 9.3
>  >  stderr: ++ 17:14:12: ocf_ver2num:516: echo 9.3
>  >  stderr: ++ 17:14:12: ocf_ver2num:516: awk '-F[.-]' '
>  >  stderr:     {for(i=1; i<=NF; i++) s=s*1000+$i; print s}
>  >  stderr:     '
>  >  stderr: + 17:14:12: ocf_version_cmp:558: v2=9003
>  >  stderr: + 17:14:12: ocf_version_cmp:559: '[' 9003 -eq 9003 ']'
>  >  stderr: + 17:14:12: ocf_version_cmp:560: return 1
>  >  stderr: + 17:14:12: pgsql_validate_all:1766: '[' 1 -eq 0 ']'
>  >  stderr: ++ 17:14:12: pgsql_validate_all:1770: get_pgsql_param 
> unix_socket_directories
>  >  stderr: ++ 17:14:12: get_pgsql_param:32: local param_name
>  >  stderr: ++ 17:14:12: get_pgsql_param:34: 
> param_name=unix_socket_directories
>  >  stderr: ++ 17:14:12: get_pgsql_param:39: perl_code='if 
> (/^\s*unix_socket_directories[\s=]+\s*(.*)$/) {
>  >  stderr:        $dir=$1;
>  >  stderr:        $dir =~ s/\s*\#.*//;
>  >  stderr:        $dir =~ s/^'\''(\S*)'\''/$1/;
>  >  stderr:        print $dir;}'
>  >  stderr: ++ 17:14:12: get_pgsql_param:41: perl -ne 'if 
> (/^\s*unix_socket_directories[\s=]+\s*(.*)$/) {
>  >  stderr:        $dir=$1;
>  >  stderr:        $dir =~ s/\s*\#.*//;
>  >  stderr:        $dir =~ s/^'\''(\S*)'\''/$1/;
>  >  stderr:        print $dir;}'
>  >  stderr: + 17:14:12: pgsql_validate_all:1770: socket_directories=
>  >  stderr: + 17:14:12: pgsql_validate_all:1771: '[' -n '' ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1782: getent passwd postgres
>  >  stderr: + 17:14:12: pgsql_validate_all:1783: '[' '!' 0 -eq 0 ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1788: ocf_is_probe
>  >  stderr: + 17:14:12: ocf_is_probe:490: '[' promote = monitor -a 0 = 
> 0 ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1791: runasowner 'test -w 
> /var/lib/pgsql/9.3/data/'
>  >  stderr: + 17:14:12: runasowner:467: local quietrun=
>  >  stderr: + 17:14:12: runasowner:468: local loglevel=-err
>  >  stderr: + 17:14:12: runasowner:469: local var
>  >  stderr: + 17:14:12: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:12: runasowner:473: case "$1" in
>  >  stderr: + 17:14:12: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:12: runasowner:473: case "$1" in
>  >  stderr: + 17:14:12: runasowner:485: ocf_run -err runuser postgres 
> -c 'cd /var/lib/pgsql/9.3/data/; test -w /var/lib/pgsql/9.3/data/'
>  >  stderr: + 17:14:12: ocf_run:413: local rc
>  >  stderr: + 17:14:12: ocf_run:414: local output
>  >  stderr: + 17:14:12: ocf_run:415: local verbose=1
>  >  stderr: + 17:14:12: ocf_run:416: local loglevel=err
>  >  stderr: + 17:14:12: ocf_run:417: local var
>  >  stderr: + 17:14:12: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:12: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:12: ocf_run:426: echo -err
>  >  stderr: ++ 17:14:12: ocf_run:426: sed -e s/-//g
>  >  stderr: + 17:14:12: ocf_run:426: loglevel=err
>  >  stderr: + 17:14:12: ocf_run:427: shift 1
>  >  stderr: + 17:14:12: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:12: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:12: ocf_run:433: runuser postgres -c 'cd 
> /var/lib/pgsql/9.3/data/; test -w /var/lib/pgsql/9.3/data/'
>  >  stderr: + 17:14:12: ocf_run:433: output=
>  >  stderr: + 17:14:12: ocf_run:434: rc=0
>  >  stderr: ++ 17:14:12: ocf_run:435: echo
>  >  stderr: + 17:14:12: ocf_run:435: output=
>  >  stderr: + 17:14:12: ocf_run:436: '[' 0 -eq 0 ']'
>  >  stderr: + 17:14:12: ocf_run:437: '[' 1 -a '!' -z '' ']'
>  >  stderr: + 17:14:12: ocf_run:440: return 0
>  >  stderr: + 17:14:12: pgsql_validate_all:1797: '[' -n '' -a '!' -n 
> '' ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1803: '[' '!' -n '' -a -n 
> '' ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1809: is_replication
>  >  stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a 
> sync '!=' slave ']'
>  >  stderr: + 17:14:12: is_replication:1288: return 0
>  >  stderr: ++ 17:14:12: pgsql_validate_all:1810: printf '9.3\n9.1'
>  >  stderr: ++ 17:14:12: pgsql_validate_all:1810: sort -n
>  >  stderr: ++ 17:14:12: pgsql_validate_all:1810: head -1
>  >  stderr: + 17:14:12: pgsql_validate_all:1810: '[' 9.1 '!=' 9.1 ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1814: '[' '!' -n 
> 10.200.200.233 ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1820: is_replication
>  >  stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a 
> sync '!=' slave ']'
>  >  stderr: + 17:14:12: is_replication:1288: return 0
>  >  stderr: + 17:14:12: pgsql_validate_all:1821: ocf_is_ms
>  >  stderr: + 17:14:12: ocf_is_ms:504: '[' '!' -z 1 ']'
>  >  stderr: + 17:14:12: ocf_is_ms:504: '[' 1 -gt 0 ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1825: '[' '!' sync = sync 
> -a '!' sync = async ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1829: '[' '!' -n 'pg1 pg2' ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1833: '[' 0 -eq 0 ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1834: 
> rep_mode_string='include '\''/var/lib/pgsql/9.3/tmp//rep_mode.conf'\'' 
> # added by pgsql RA'
>  >  stderr: + 17:14:12: pgsql_validate_all:1835: '[' sync = sync ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1836: grep -q 'include 
> '\''/var/lib/pgsql/9.3/tmp//rep_mode.conf'\'' # added by pgsql RA' 
> /var/lib/pgsql/9.3/data//postgresql.conf
>  >  stderr: + 17:14:12: pgsql_validate_all:1848: mkdir -p 
> /var/lib/pgsql/9.3/tmp/
>  >  stderr: + 17:14:12: pgsql_validate_all:1848: chown postgres 
> /var/lib/pgsql/9.3/tmp/
>  >  stderr: + 17:14:12: pgsql_validate_all:1848: chmod 700 
> /var/lib/pgsql/9.3/tmp/
>  >  stderr: + 17:14:12: pgsql_validate_all:1854: '[' sync = slave ']'
>  >  stderr: + 17:14:12: pgsql_validate_all:1861: use_replication_slot
>  >  stderr: + 17:14:12: use_replication_slot:1294: '[' -n '' ']'
>  >  stderr: + 17:14:12: use_replication_slot:1298: return 1
>  >  stderr: + 17:14:12: pgsql_validate_all:1869: return 0
>  >  stderr: + 17:14:12: 1985: rc=0
>  >  stderr: + 17:14:12: 1987: '[' promote = validate-all ']'
>  >  stderr: + 17:14:12: 1989: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:12: 2002: id -u -n
>  >  stderr: + 17:14:12: 2002: US=root
>  >  stderr: + 17:14:12: 2004: '[' root '!=' root -a root '!=' postgres 
> ']'
>  >  stderr: + 17:14:12: 2011: '[' -n '' ']'
>  >  stderr: + 17:14:12: 2016: psql_options='-p 5432 -U postgres 
> template1'
>  >  stderr: + 17:14:12: 2019: '[' -n '' ']'
>  >  stderr: + 17:14:12: 2022: '[' -n '' ']'
>  >  stderr: + 17:14:12: 2027: '[' -n 5432 ']'
>  >  stderr: + 17:14:12: 2028: export PGPORT=5432
>  >  stderr: + 17:14:12: 2028: PGPORT=5432
>  >  stderr: + 17:14:12: 2031: '[' -n /usr/lib ']'
>  >  stderr: + 17:14:12: 2032: '[' -n '' ']'
>  >  stderr: + 17:14:12: 2035: export LD_LIBRARY_PATH=/usr/lib
>  >  stderr: + 17:14:12: 2035: LD_LIBRARY_PATH=/usr/lib
>  >  stderr: + 17:14:12: 2041: case "$1" in
>  >  stderr: + 17:14:12: 2057: pgsql_promote
>  >  stderr: + 17:14:12: pgsql_promote:665: local target
>  >  stderr: + 17:14:12: pgsql_promote:666: local rc
>  >  stderr: + 17:14:12: pgsql_promote:668: is_replication
>  >  stderr: + 17:14:12: is_replication:1287: '[' sync '!=' none -a 
> sync '!=' slave ']'
>  >  stderr: + 17:14:12: is_replication:1288: return 0
>  >  stderr: + 17:14:12: pgsql_promote:672: rm -f 
> '/var/lib/pgsql/9.3/tmp//xlog_note.*'
>  >  stderr: + 17:14:12: pgsql_promote:674: for target in '$NODE_LIST'
>  >  stderr: + 17:14:12: pgsql_promote:675: '[' pg1 = pm1 ']'
>  >  stderr: + 17:14:12: pgsql_promote:676: change_data_status pg1 
> DISCONNECT
>  >  stderr: + 17:14:12: change_data_status:1569: local output
>  >  stderr: + 17:14:12: change_data_status:1571: node_exist pg1
>  >  stderr: + 17:14:12: node_exist:1720: print_crm_mon
>  >  stderr: + 17:14:12: node_exist:1720: tr '[A-Z]' '[a-z]'
>  >  stderr: + 17:14:12: node_exist:1720: grep -q '^node pg1'
>  >  stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>  >  stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0 crm_mon 
> -n1
>  >  stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>  >  stderr: ++ 17:14:12: exec_with_retry:1690: local output
>  >  stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>  >  stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:12: exec_with_retry:1696: shift
>  >  stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>  >  stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>  >  stderr: ++ 17:14:12: exec_with_retry:1699: output='Last updated: 
> Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>  >  stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last updated: 
> Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>  >  stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last 
> updated: Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 
> 16:07:56 2015 by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated: Mon 
> Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 by 
> root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave\n'
>  >  stderr: + 17:14:12: change_data_status:1572: return 0
>  >  stderr: + 17:14:12: pgsql_promote:677: change_master_score pg1 
> -INFINITY
>  >  stderr: + 17:14:12: change_master_score:1604: local instance
>  >  stderr: + 17:14:12: change_master_score:1606: is_node_online pg1
>  >  stderr: + 17:14:12: is_node_online:1716: print_crm_mon
>  >  stderr: + 17:14:12: is_node_online:1716: tr '[A-Z]' '[a-z]'
>  >  stderr: + 17:14:12: is_node_online:1716: grep -e '^node pg1 ' -e 
> '^node pg1:'
>  >  stderr: + 17:14:12: is_node_online:1716: grep -q -v offline
>  >  stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>  >  stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0 crm_mon 
> -n1
>  >  stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>  >  stderr: ++ 17:14:12: exec_with_retry:1690: local output
>  >  stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>  >  stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:12: exec_with_retry:1696: shift
>  >  stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>  >  stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>  >  stderr: ++ 17:14:12: exec_with_retry:1699: output='Last updated: 
> Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>  >  stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last updated: 
> Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>  >  stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last 
> updated: Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 
> 16:07:56 2015 by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated: Mon 
> Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 by 
> root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave\n'
>  >  stderr: + 17:14:12: change_master_score:1607: return 0
>  >  stderr: + 17:14:12: pgsql_promote:674: for target in '$NODE_LIST'
>  >  stderr: + 17:14:12: pgsql_promote:675: '[' pg2 = pm1 ']'
>  >  stderr: + 17:14:12: pgsql_promote:676: change_data_status pg2 
> DISCONNECT
>  >  stderr: + 17:14:12: change_data_status:1569: local output
>  >  stderr: + 17:14:12: change_data_status:1571: node_exist pg2
>  >  stderr: + 17:14:12: node_exist:1720: print_crm_mon
>  >  stderr: + 17:14:12: node_exist:1720: tr '[A-Z]' '[a-z]'
>  >  stderr: + 17:14:12: node_exist:1720: grep -q '^node pg2'
>  >  stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>  >  stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0 crm_mon 
> -n1
>  >  stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>  >  stderr: ++ 17:14:12: exec_with_retry:1690: local output
>  >  stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>  >  stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:12: exec_with_retry:1696: shift
>  >  stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>  >  stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>  >  stderr: ++ 17:14:12: exec_with_retry:1699: output='Last updated: 
> Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>  >  stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last updated: 
> Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>  >  stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last 
> updated: Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 
> 16:07:56 2015 by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated: Mon 
> Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 by 
> root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave\n'
>  >  stderr: + 17:14:12: change_data_status:1572: return 0
>  >  stderr: + 17:14:12: pgsql_promote:677: change_master_score pg2 
> -INFINITY
>  >  stderr: + 17:14:12: change_master_score:1604: local instance
>  >  stderr: + 17:14:12: change_master_score:1606: is_node_online pg2
>  >  stderr: + 17:14:12: is_node_online:1716: grep -q -v offline
>  >  stderr: + 17:14:12: is_node_online:1716: print_crm_mon
>  >  stderr: + 17:14:12: is_node_online:1716: tr '[A-Z]' '[a-z]'
>  >  stderr: + 17:14:12: is_node_online:1716: grep -e '^node pg2 ' -e 
> '^node pg2:'
>  >  stderr: + 17:14:12: print_crm_mon:1924: '[' -z '' ']'
>  >  stderr: ++ 17:14:12: print_crm_mon:1925: exec_with_retry 0 crm_mon 
> -n1
>  >  stderr: ++ 17:14:12: exec_with_retry:1689: local count=86400
>  >  stderr: ++ 17:14:12: exec_with_retry:1690: local output
>  >  stderr: ++ 17:14:12: exec_with_retry:1691: local rc
>  >  stderr: ++ 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:12: exec_with_retry:1696: shift
>  >  stderr: ++ 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>  >  stderr: +++ 17:14:12: exec_with_retry:1699: crm_mon -n1
>  >  stderr: ++ 17:14:12: exec_with_retry:1699: output='Last updated: 
> Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:12: exec_with_retry:1700: rc=0
>  >  stderr: ++ 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:12: exec_with_retry:1706: printf 'Last updated: 
> Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:12: exec_with_retry:1707: return 0
>  >  stderr: + 17:14:12: print_crm_mon:1925: CRM_MON_OUTPUT='Last 
> updated: Mon Oct 12 17:14:12 2015        Last change: Mon Oct 12 
> 16:07:56 2015 by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: + 17:14:12: print_crm_mon:1927: printf 'Last updated: Mon 
> Oct 12 17:14:12 2015        Last change: Mon Oct 12 16:07:56 2015 by 
> root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave\n'
>  >  stderr: + 17:14:12: change_master_score:1607: return 0
>  >  stderr: + 17:14:12: pgsql_promote:680: ocf_log info 'Creating 
> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>  >  stderr: + 17:14:12: ocf_log:321: '[' 2 -lt 2 ']'
>  >  stderr: + 17:14:12: ocf_log:325: __OCF_PRIO=info
>  >  stderr: + 17:14:12: ocf_log:326: shift
>  >  stderr: + 17:14:12: ocf_log:327: __OCF_MSG='Creating 
> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>  >  stderr: + 17:14:12: ocf_log:329: case "${__OCF_PRIO}" in
>  >  stderr: + 17:14:12: ocf_log:333: __OCF_PRIO=INFO
>  >  stderr: + 17:14:12: ocf_log:338: '[' INFO = DEBUG ']'
>  >  stderr: + 17:14:12: ocf_log:341: ha_log 'INFO: Creating 
> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>  >  stderr: + 17:14:12: ha_log:253: __ha_log 'INFO: Creating 
> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>  >  stderr: + 17:14:12: __ha_log:185: local ignore_stderr=false
>  >  stderr: + 17:14:12: __ha_log:186: local loglevel
>  >  stderr: + 17:14:12: __ha_log:188: '[' 'xINFO: Creating 
> /var/lib/pgsql/9.3/tmp//PGSQL.lock.' = x--ignore-stderr ']'
>  >  stderr: + 17:14:12: __ha_log:190: '[' none = '' ']'
>  >  stderr: + 17:14:12: __ha_log:192: tty
>  >  stderr: + 17:14:12: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>  >  stderr: + 17:14:12: __ha_log:195: '[' false = true ']'
>  >  stderr: + 17:14:12: __ha_log:199: '[' '' ']'
>  >  stderr: + 17:14:12: __ha_log:202: echo 'INFO: Creating 
> /var/lib/pgsql/9.3/tmp//PGSQL.lock.'
>  >  stderr: INFO: Creating /var/lib/pgsql/9.3/tmp//PGSQL.lock.
>  >  stderr: + 17:14:12: __ha_log:204: return 0
>  >  stderr: + 17:14:12: pgsql_promote:681: touch 
> /var/lib/pgsql/9.3/tmp//PGSQL.lock
>  >  stderr: + 17:14:12: pgsql_promote:682: show_master_baseline
>  >  stderr: + 17:14:12: show_master_baseline:1438: local rc
>  >  stderr: + 17:14:12: show_master_baseline:1439: local location
>  >  stderr: ++ 17:14:12: show_master_baseline:1441: get_my_location
>  >  stderr: ++ 17:14:12: get_my_location:1389: local rc
>  >  stderr: ++ 17:14:12: get_my_location:1390: local output
>  >  stderr: ++ 17:14:12: get_my_location:1391: local replay_loc
>  >  stderr: ++ 17:14:12: get_my_location:1392: local receive_loc
>  >  stderr: ++ 17:14:12: get_my_location:1393: local output1
>  >  stderr: ++ 17:14:12: get_my_location:1394: local output2
>  >  stderr: ++ 17:14:12: get_my_location:1395: local log1
>  >  stderr: ++ 17:14:12: get_my_location:1396: local log2
>  >  stderr: ++ 17:14:12: get_my_location:1397: local newer_location
>  >  stderr: +++ 17:14:12: get_my_location:1399: exec_sql ''
>  >  stderr: +++ 17:14:12: exec_sql:518: local sql=
>  >  stderr: +++ 17:14:12: exec_sql:519: local output
>  >  stderr: +++ 17:14:12: exec_sql:520: local rc
>  >  stderr: ++++ 17:14:12: exec_sql:524: runuser postgres -c 'cd 
> /var/lib/pgsql/9.3/data/; /usr/pgsql-9.3/bin/psql -p 5432 -U postgres 
> template1 -U postgres                 -Atc ""'
>  >  stderr: +++ 17:14:12: exec_sql:524: output=
>  >  stderr: +++ 17:14:12: exec_sql:525: rc=0
>  >  stderr: +++ 17:14:12: exec_sql:527: echo
>  >  stderr: +++ 17:14:12: exec_sql:528: return 0
>  >  stderr: ++ 17:14:12: get_my_location:1399: output=
>  >  stderr: ++ 17:14:12: get_my_location:1400: rc=0
>  >  stderr: ++ 17:14:12: get_my_location:1402: '[' 0 -ne 0 ']'
>  >  stderr: +++ 17:14:12: get_my_location:1406: cut -d '|' -f 1
>  >  stderr: +++ 17:14:12: get_my_location:1406: echo
>  >  stderr: ++ 17:14:12: get_my_location:1406: replay_loc=
>  >  stderr: +++ 17:14:12: get_my_location:1407: cut -d '|' -f 2
>  >  stderr: +++ 17:14:12: get_my_location:1407: echo
>  >  stderr: ++ 17:14:12: get_my_location:1407: receive_loc=
>  >  stderr: +++ 17:14:12: get_my_location:1409: echo ''
>  >  stderr: +++ 17:14:12: get_my_location:1409: cut -d / -f 1
>  >  stderr: ++ 17:14:12: get_my_location:1409: output1=
>  >  stderr: +++ 17:14:12: get_my_location:1410: echo ''
>  >  stderr: +++ 17:14:12: get_my_location:1410: cut -d / -f 2
>  >  stderr: ++ 17:14:12: get_my_location:1410: output2=
>  >  stderr: +++ 17:14:12: get_my_location:1411: printf '%08s\n'
>  >  stderr: +++ 17:14:12: get_my_location:1411: sed 's/ /0/g'
>  >  stderr: ++ 17:14:12: get_my_location:1411: log1=00000000
>  >  stderr: +++ 17:14:12: get_my_location:1412: printf '%08s\n'
>  >  stderr: +++ 17:14:12: get_my_location:1412: sed 's/ /0/g'
>  >  stderr: ++ 17:14:12: get_my_location:1412: log2=00000000
>  >  stderr: ++ 17:14:12: get_my_location:1413: 
> replay_loc=0000000000000000
>  >  stderr: +++ 17:14:12: get_my_location:1415: cut -d / -f 1
>  >  stderr: +++ 17:14:12: get_my_location:1415: echo ''
>  >  stderr: ++ 17:14:12: get_my_location:1415: output1=
>  >  stderr: +++ 17:14:12: get_my_location:1416: cut -d / -f 2
>  >  stderr: +++ 17:14:12: get_my_location:1416: echo ''
>  >  stderr: ++ 17:14:12: get_my_location:1416: output2=
>  >  stderr: +++ 17:14:12: get_my_location:1417: sed 's/ /0/g'
>  >  stderr: +++ 17:14:12: get_my_location:1417: printf '%08s\n'
>  >  stderr: ++ 17:14:12: get_my_location:1417: log1=00000000
>  >  stderr: +++ 17:14:12: get_my_location:1418: sed 's/ /0/g'
>  >  stderr: +++ 17:14:12: get_my_location:1418: printf '%08s\n'
>  >  stderr: ++ 17:14:12: get_my_location:1418: log2=00000000
>  >  stderr: ++ 17:14:12: get_my_location:1419: 
> receive_loc=0000000000000000
>  >  stderr: +++ 17:14:12: get_my_location:1421: printf 
> '0000000000000000\n0000000000000000'
>  >  stderr: +++ 17:14:12: get_my_location:1421: sort -r
>  >  stderr: +++ 17:14:12: get_my_location:1421: head -1
>  >  stderr: ++ 17:14:12: get_my_location:1421: 
> newer_location=0000000000000000
>  >  stderr: ++ 17:14:12: get_my_location:1422: echo 0000000000000000
>  >  stderr: ++ 17:14:12: get_my_location:1423: return 0
>  >  stderr: + 17:14:12: show_master_baseline:1441: 
> location=0000000000000000
>  >  stderr: + 17:14:12: show_master_baseline:1442: ocf_log info 'My 
> master baseline : 0000000000000000.'
>  >  stderr: + 17:14:12: ocf_log:321: '[' 2 -lt 2 ']'
>  >  stderr: + 17:14:12: ocf_log:325: __OCF_PRIO=info
>  >  stderr: + 17:14:12: ocf_log:326: shift
>  >  stderr: + 17:14:12: ocf_log:327: __OCF_MSG='My master baseline : 
> 0000000000000000.'
>  >  stderr: + 17:14:12: ocf_log:329: case "${__OCF_PRIO}" in
>  >  stderr: + 17:14:12: ocf_log:333: __OCF_PRIO=INFO
>  >  stderr: + 17:14:12: ocf_log:338: '[' INFO = DEBUG ']'
>  >  stderr: + 17:14:12: ocf_log:341: ha_log 'INFO: My master baseline 
> : 0000000000000000.'
>  >  stderr: + 17:14:12: ha_log:253: __ha_log 'INFO: My master baseline 
> : 0000000000000000.'
>  >  stderr: + 17:14:12: __ha_log:185: local ignore_stderr=false
>  >  stderr: + 17:14:12: __ha_log:186: local loglevel
>  >  stderr: + 17:14:12: __ha_log:188: '[' 'xINFO: My master baseline : 
> 0000000000000000.' = x--ignore-stderr ']'
>  >  stderr: + 17:14:12: __ha_log:190: '[' none = '' ']'
>  >  stderr: + 17:14:12: __ha_log:192: tty
>  >  stderr: + 17:14:12: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>  >  stderr: + 17:14:12: __ha_log:195: '[' false = true ']'
>  >  stderr: + 17:14:12: __ha_log:199: '[' '' ']'
>  >  stderr: + 17:14:12: __ha_log:202: echo 'INFO: My master baseline : 
> 0000000000000000.'
>  >  stderr: INFO: My master baseline : 0000000000000000.
>  >  stderr: + 17:14:12: __ha_log:204: return 0
>  >  stderr: + 17:14:12: show_master_baseline:1443: exec_with_retry 0 
> /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-master-baseline -v 
> 0000000000000000
>  >  stderr: + 17:14:12: exec_with_retry:1689: local count=86400
>  >  stderr: + 17:14:12: exec_with_retry:1690: local output
>  >  stderr: + 17:14:12: exec_with_retry:1691: local rc
>  >  stderr: + 17:14:12: exec_with_retry:1693: '[' 0 -ne 0 ']'
>  >  stderr: + 17:14:12: exec_with_retry:1696: shift
>  >  stderr: + 17:14:12: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>  >  stderr: ++ 17:14:12: exec_with_retry:1699: /usr/sbin/crm_attribute 
> -l reboot -N pm1 -n pgsql-master-baseline -v 0000000000000000
>  >  stderr: + 17:14:12: exec_with_retry:1699: output=
>  >  stderr: + 17:14:12: exec_with_retry:1700: rc=0
>  >  stderr: + 17:14:12: exec_with_retry:1701: '[' 0 -ne 0 ']'
>  >  stderr: + 17:14:12: exec_with_retry:1706: printf ''
>  >  stderr: + 17:14:12: exec_with_retry:1707: return 0
>  >  stderr: + 17:14:12: pgsql_promote:684: ocf_is_true true
>  >  stderr: + 17:14:12: ocf_is_true:101: case "$1" in
>  >  stderr: + 17:14:12: ocf_is_true:102: true
>  >  stderr: + 17:14:12: pgsql_promote:685: ocf_log info 'Restarting 
> PostgreSQL instead of promote.'
>  >  stderr: + 17:14:12: ocf_log:321: '[' 2 -lt 2 ']'
>  >  stderr: + 17:14:12: ocf_log:325: __OCF_PRIO=info
>  >  stderr: + 17:14:12: ocf_log:326: shift
>  >  stderr: + 17:14:12: ocf_log:327: __OCF_MSG='Restarting PostgreSQL 
> instead of promote.'
>  >  stderr: + 17:14:12: ocf_log:329: case "${__OCF_PRIO}" in
>  >  stderr: + 17:14:12: ocf_log:333: __OCF_PRIO=INFO
>  >  stderr: + 17:14:12: ocf_log:338: '[' INFO = DEBUG ']'
>  >  stderr: + 17:14:12: ocf_log:341: ha_log 'INFO: Restarting 
> PostgreSQL instead of promote.'
>  >  stderr: + 17:14:12: ha_log:253: __ha_log 'INFO: Restarting 
> PostgreSQL instead of promote.'
>  >  stderr: + 17:14:12: __ha_log:185: local ignore_stderr=false
>  >  stderr: + 17:14:12: __ha_log:186: local loglevel
>  >  stderr: + 17:14:12: __ha_log:188: '[' 'xINFO: Restarting 
> PostgreSQL instead of promote.' = x--ignore-stderr ']'
>  >  stderr: + 17:14:12: __ha_log:190: '[' none = '' ']'
>  >  stderr: + 17:14:12: __ha_log:192: tty
>  >  stderr: + 17:14:12: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>  >  stderr: + 17:14:12: __ha_log:195: '[' false = true ']'
>  >  stderr: + 17:14:12: __ha_log:199: '[' '' ']'
>  >  stderr: + 17:14:12: __ha_log:202: echo 'INFO: Restarting 
> PostgreSQL instead of promote.'
>  >  stderr: INFO: Restarting PostgreSQL instead of promote.
>  >  stderr: + 17:14:12: __ha_log:204: return 0
>  >  stderr: + 17:14:12: pgsql_promote:687: pgsql_real_stop slave
>  >  stderr: + 17:14:12: pgsql_real_stop:758: local rc
>  >  stderr: + 17:14:12: pgsql_real_stop:759: local count
>  >  stderr: + 17:14:12: pgsql_real_stop:760: local stop_escalate
>  >  stderr: + 17:14:12: pgsql_real_stop:762: ocf_is_true false
>  >  stderr: + 17:14:12: ocf_is_true:101: case "$1" in
>  >  stderr: + 17:14:12: ocf_is_true:103: false
>  >  stderr: + 17:14:12: pgsql_real_stop:766: pgsql_status
>  >  stderr: + 17:14:12: pgsql_status:866: '[' -f 
> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>  >  stderr: ++ 17:14:12: pgsql_status:868: head -n 1 
> /var/lib/pgsql/9.3/data//postmaster.pid
>  >  stderr: + 17:14:12: pgsql_status:868: PID=6046
>  >  stderr: + 17:14:12: pgsql_status:869: runasowner 'kill -s 0 6046 
> >/dev/null 2>&1'
>  >  stderr: + 17:14:12: runasowner:467: local quietrun=
>  >  stderr: + 17:14:12: runasowner:468: local loglevel=-err
>  >  stderr: + 17:14:12: runasowner:469: local var
>  >  stderr: + 17:14:12: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:12: runasowner:473: case "$1" in
>  >  stderr: + 17:14:12: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:12: runasowner:473: case "$1" in
>  >  stderr: + 17:14:12: runasowner:485: ocf_run -err runuser postgres 
> -c 'cd /var/lib/pgsql/9.3/data/; kill -s 0 6046 >/dev/null 2>&1'
>  >  stderr: + 17:14:12: ocf_run:413: local rc
>  >  stderr: + 17:14:12: ocf_run:414: local output
>  >  stderr: + 17:14:12: ocf_run:415: local verbose=1
>  >  stderr: + 17:14:12: ocf_run:416: local loglevel=err
>  >  stderr: + 17:14:12: ocf_run:417: local var
>  >  stderr: + 17:14:12: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:12: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:12: ocf_run:426: echo -err
>  >  stderr: ++ 17:14:12: ocf_run:426: sed -e s/-//g
>  >  stderr: + 17:14:12: ocf_run:426: loglevel=err
>  >  stderr: + 17:14:12: ocf_run:427: shift 1
>  >  stderr: + 17:14:12: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:12: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:12: ocf_run:433: runuser postgres -c 'cd 
> /var/lib/pgsql/9.3/data/; kill -s 0 6046 >/dev/null 2>&1'
>  >  stderr: + 17:14:12: ocf_run:433: output=
>  >  stderr: + 17:14:12: ocf_run:434: rc=0
>  >  stderr: ++ 17:14:12: ocf_run:435: echo
>  >  stderr: + 17:14:12: ocf_run:435: output=
>  >  stderr: + 17:14:12: ocf_run:436: '[' 0 -eq 0 ']'
>  >  stderr: + 17:14:12: ocf_run:437: '[' 1 -a '!' -z '' ']'
>  >  stderr: + 17:14:12: ocf_run:440: return 0
>  >  stderr: + 17:14:12: pgsql_status:870: return 0
>  >  stderr: + 17:14:12: pgsql_real_stop:772: stop_escalate=30
>  >  stderr: + 17:14:12: pgsql_real_stop:773: '[' slave = slave ']'
>  >  stderr: + 17:14:12: pgsql_real_stop:774: stop_escalate=30
>  >  stderr: + 17:14:12: pgsql_real_stop:778: '[' 30 -gt 0 ']'
>  >  stderr: + 17:14:12: pgsql_real_stop:779: runasowner 
> '/usr/pgsql-9.3/bin/pg_ctl -D /var/lib/pgsql/9.3/data/ stop -m fast'
>  >  stderr: + 17:14:12: runasowner:467: local quietrun=
>  >  stderr: + 17:14:12: runasowner:468: local loglevel=-err
>  >  stderr: + 17:14:12: runasowner:469: local var
>  >  stderr: + 17:14:12: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:12: runasowner:473: case "$1" in
>  >  stderr: + 17:14:12: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:12: runasowner:473: case "$1" in
>  >  stderr: + 17:14:12: runasowner:485: ocf_run -err runuser postgres 
> -c 'cd /var/lib/pgsql/9.3/data/; /usr/pgsql-9.3/bin/pg_ctl -D 
> /var/lib/pgsql/9.3/data/ stop -m fast'
>  >  stderr: + 17:14:12: ocf_run:413: local rc
>  >  stderr: + 17:14:12: ocf_run:414: local output
>  >  stderr: + 17:14:12: ocf_run:415: local verbose=1
>  >  stderr: + 17:14:12: ocf_run:416: local loglevel=err
>  >  stderr: + 17:14:12: ocf_run:417: local var
>  >  stderr: + 17:14:12: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:12: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:12: ocf_run:426: echo -err
>  >  stderr: ++ 17:14:12: ocf_run:426: sed -e s/-//g
>  >  stderr: + 17:14:12: ocf_run:426: loglevel=err
>  >  stderr: + 17:14:12: ocf_run:427: shift 1
>  >  stderr: + 17:14:12: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:12: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:12: ocf_run:433: runuser postgres -c 'cd 
> /var/lib/pgsql/9.3/data/; /usr/pgsql-9.3/bin/pg_ctl -D 
> /var/lib/pgsql/9.3/data/ stop -m fast'
>  >  stderr: + 17:14:13: ocf_run:433: output='waiting for server to 
> shut down.... done
>  >  stderr: server stopped'
>  >  stderr: + 17:14:13: ocf_run:434: rc=0
>  >  stderr: ++ 17:14:13: ocf_run:435: echo waiting for server to shut 
> down.... done server stopped
>  >  stderr: + 17:14:13: ocf_run:435: output='waiting for server to 
> shut down.... done server stopped'
>  >  stderr: + 17:14:13: ocf_run:436: '[' 0 -eq 0 ']'
>  >  stderr: + 17:14:13: ocf_run:437: '[' 1 -a '!' -z 'waiting for 
> server to shut down.... done server stopped' ']'
>  >  stderr: + 17:14:13: ocf_run:438: ocf_log info 'waiting for server 
> to shut down.... done server stopped'
>  >  stderr: + 17:14:13: ocf_log:321: '[' 2 -lt 2 ']'
>  >  stderr: + 17:14:13: ocf_log:325: __OCF_PRIO=info
>  >  stderr: + 17:14:13: ocf_log:326: shift
>  >  stderr: + 17:14:13: ocf_log:327: __OCF_MSG='waiting for server to 
> shut down.... done server stopped'
>  >  stderr: + 17:14:13: ocf_log:329: case "${__OCF_PRIO}" in
>  >  stderr: + 17:14:13: ocf_log:333: __OCF_PRIO=INFO
>  >  stderr: + 17:14:13: ocf_log:338: '[' INFO = DEBUG ']'
>  >  stderr: + 17:14:13: ocf_log:341: ha_log 'INFO: waiting for server 
> to shut down.... done server stopped'
>  >  stderr: + 17:14:13: ha_log:253: __ha_log 'INFO: waiting for server 
> to shut down.... done server stopped'
>  >  stderr: + 17:14:13: __ha_log:185: local ignore_stderr=false
>  >  stderr: + 17:14:13: __ha_log:186: local loglevel
>  >  stderr: + 17:14:13: __ha_log:188: '[' 'xINFO: waiting for server 
> to shut down.... done server stopped' = x--ignore-stderr ']'
>  >  stderr: + 17:14:13: __ha_log:190: '[' none = '' ']'
>  >  stderr: + 17:14:13: __ha_log:192: tty
>  >  stderr: + 17:14:13: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>  >  stderr: + 17:14:13: __ha_log:195: '[' false = true ']'
>  >  stderr: + 17:14:13: __ha_log:199: '[' '' ']'
>  >  stderr: + 17:14:13: __ha_log:202: echo 'INFO: waiting for server 
> to shut down.... done server stopped'
>  >  stderr: INFO: waiting for server to shut down.... done server stopped
>  >  stderr: + 17:14:13: __ha_log:204: return 0
>  >  stderr: + 17:14:13: ocf_run:440: return 0
>  >  stderr: + 17:14:13: pgsql_real_stop:783: count=0
>  >  stderr: + 17:14:13: pgsql_real_stop:784: '[' 0 -lt 30 ']'
>  >  stderr: + 17:14:13: pgsql_real_stop:786: pgsql_status
>  >  stderr: + 17:14:13: pgsql_status:866: '[' -f 
> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>  >  stderr: + 17:14:13: pgsql_status:874: false
>  >  stderr: + 17:14:13: pgsql_real_stop:789: break
>  >  stderr: + 17:14:13: pgsql_real_stop:795: pgsql_status
>  >  stderr: + 17:14:13: pgsql_status:866: '[' -f 
> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>  >  stderr: + 17:14:13: pgsql_status:874: false
>  >  stderr: + 17:14:13: pgsql_real_stop:802: :
>  >  stderr: + 17:14:13: pgsql_real_stop:804: pgsql_real_monitor
>  >  stderr: + 17:14:13: pgsql_real_monitor:905: local loglevel
>  >  stderr: + 17:14:13: pgsql_real_monitor:906: local rc
>  >  stderr: + 17:14:13: pgsql_real_monitor:907: local output
>  >  stderr: + 17:14:13: pgsql_real_monitor:910: loglevel=err
>  >  stderr: + 17:14:13: pgsql_real_monitor:912: pgsql_status
>  >  stderr: + 17:14:13: pgsql_status:866: '[' -f 
> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>  >  stderr: + 17:14:13: pgsql_status:874: false
>  >  stderr: + 17:14:13: pgsql_real_monitor:914: ocf_log info 
> 'PostgreSQL is down'
>  >  stderr: + 17:14:13: ocf_log:321: '[' 2 -lt 2 ']'
>  >  stderr: + 17:14:13: ocf_log:325: __OCF_PRIO=info
>  >  stderr: + 17:14:13: ocf_log:326: shift
>  >  stderr: + 17:14:13: ocf_log:327: __OCF_MSG='PostgreSQL is down'
>  >  stderr: + 17:14:13: ocf_log:329: case "${__OCF_PRIO}" in
>  >  stderr: + 17:14:13: ocf_log:333: __OCF_PRIO=INFO
>  >  stderr: + 17:14:13: ocf_log:338: '[' INFO = DEBUG ']'
>  >  stderr: + 17:14:13: ocf_log:341: ha_log 'INFO: PostgreSQL is down'
>  >  stderr: + 17:14:13: ha_log:253: __ha_log 'INFO: PostgreSQL is down'
>  >  stderr: + 17:14:13: __ha_log:185: local ignore_stderr=false
>  >  stderr: + 17:14:13: __ha_log:186: local loglevel
>  >  stderr: + 17:14:13: __ha_log:188: '[' 'xINFO: PostgreSQL is down' 
> = x--ignore-stderr ']'
>  >  stderr: + 17:14:13: __ha_log:190: '[' none = '' ']'
>  >  stderr: + 17:14:13: __ha_log:192: tty
>  >  stderr: + 17:14:13: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>  >  stderr: + 17:14:13: __ha_log:195: '[' false = true ']'
>  >  stderr: + 17:14:13: __ha_log:199: '[' '' ']'
>  >  stderr: + 17:14:13: __ha_log:202: echo 'INFO: PostgreSQL is down'
>  >  stderr: INFO: PostgreSQL is down
>  >  stderr: + 17:14:13: __ha_log:204: return 0
>  >  stderr: + 17:14:13: pgsql_real_monitor:915: return 7
>  >  stderr: + 17:14:13: pgsql_real_stop:805: rc=7
>  >  stderr: + 17:14:13: pgsql_real_stop:806: '[' 7 -eq 7 ']'
>  >  stderr: + 17:14:13: pgsql_real_stop:808: break
>  >  stderr: + 17:14:13: pgsql_real_stop:815: rm -f 
> /var/lib/pgsql/9.3/data//postmaster.pid
>  >  stderr: + 17:14:13: pgsql_real_stop:817: '[' slave = master -a '' 
> = ' ' ']'
>  >  stderr: + 17:14:13: pgsql_real_stop:821: return 0
>  >  stderr: + 17:14:13: pgsql_promote:688: rm -f 
> /var/lib/pgsql/9.3/data//recovery.conf
>  >  stderr: + 17:14:13: pgsql_promote:689: pgsql_real_start
>  >  stderr: + 17:14:13: pgsql_real_start:534: local pgctl_options
>  >  stderr: + 17:14:13: pgsql_real_start:535: local postgres_options
>  >  stderr: + 17:14:13: pgsql_real_start:536: local rc
>  >  stderr: + 17:14:13: pgsql_real_start:538: pgsql_status
>  >  stderr: + 17:14:13: pgsql_status:866: '[' -f 
> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>  >  stderr: + 17:14:13: pgsql_status:874: false
>  >  stderr: + 17:14:13: pgsql_real_start:548: rm -f 
> /var/lib/pgsql/9.3/data//postmaster.pid
>  >  stderr: + 17:14:13: pgsql_real_start:551: '[' -f 
> /var/lib/pgsql/9.3/data//backup_label ']'
>  >  stderr: + 17:14:13: pgsql_real_start:557: check_log_file /dev/null
>  >  stderr: + 17:14:13: check_log_file:1878: '[' '!' -f /dev/null ']'
>  >  stderr: + 17:14:13: check_log_file:1880: touch /dev/null
>  >  stderr: ++ 17:14:13: check_log_file:1881: getent passwd postgres
>  >  stderr: ++ 17:14:13: check_log_file:1881: cut -d : -f 4
>  >  stderr: + 17:14:13: check_log_file:1881: chown postgres:26 /dev/null
>  >  stderr: + 17:14:13: check_log_file:1885: runasowner 'test -w 
> /dev/null'
>  >  stderr: + 17:14:13: runasowner:467: local quietrun=
>  >  stderr: + 17:14:13: runasowner:468: local loglevel=-err
>  >  stderr: + 17:14:13: runasowner:469: local var
>  >  stderr: + 17:14:13: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:13: runasowner:473: case "$1" in
>  >  stderr: + 17:14:13: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:13: runasowner:473: case "$1" in
>  >  stderr: + 17:14:13: runasowner:485: ocf_run -err runuser postgres 
> -c 'cd /var/lib/pgsql/9.3/data/; test -w /dev/null'
>  >  stderr: + 17:14:13: ocf_run:413: local rc
>  >  stderr: + 17:14:13: ocf_run:414: local output
>  >  stderr: + 17:14:13: ocf_run:415: local verbose=1
>  >  stderr: + 17:14:13: ocf_run:416: local loglevel=err
>  >  stderr: + 17:14:13: ocf_run:417: local var
>  >  stderr: + 17:14:13: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:13: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:13: ocf_run:426: echo -err
>  >  stderr: ++ 17:14:13: ocf_run:426: sed -e s/-//g
>  >  stderr: + 17:14:13: ocf_run:426: loglevel=err
>  >  stderr: + 17:14:13: ocf_run:427: shift 1
>  >  stderr: + 17:14:13: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:13: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:13: ocf_run:433: runuser postgres -c 'cd 
> /var/lib/pgsql/9.3/data/; test -w /dev/null'
>  >  stderr: + 17:14:13: ocf_run:433: output=
>  >  stderr: + 17:14:13: ocf_run:434: rc=0
>  >  stderr: ++ 17:14:13: ocf_run:435: echo
>  >  stderr: + 17:14:13: ocf_run:435: output=
>  >  stderr: + 17:14:13: ocf_run:436: '[' 0 -eq 0 ']'
>  >  stderr: + 17:14:13: ocf_run:437: '[' 1 -a '!' -z '' ']'
>  >  stderr: + 17:14:13: ocf_run:440: return 0
>  >  stderr: + 17:14:13: check_log_file:1890: return 0
>  >  stderr: + 17:14:13: pgsql_real_start:564: '[' -n '' ']'
>  >  stderr: + 17:14:13: pgsql_real_start:569: '[' sync = slave ']'
>  >  stderr: + 17:14:13: pgsql_real_start:575: pgctl_options=' -D 
> /var/lib/pgsql/9.3/data/ -l /dev/null'
>  >  stderr: + 17:14:13: pgsql_real_start:578: postgres_options='-c 
> config_file=/var/lib/pgsql/9.3/data//postgresql.conf'
>  >  stderr: + 17:14:13: pgsql_real_start:580: '[' -n '' ']'
>  >  stderr: + 17:14:13: pgsql_real_start:583: '[' -n '' ']'
>  >  stderr: + 17:14:13: pgsql_real_start:588: pgctl_options=' -D 
> /var/lib/pgsql/9.3/data/ -l /dev/null -o '\''-c 
> config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\'''
>  >  stderr: + 17:14:13: pgsql_real_start:591: runasowner 'unset 
> PGUSER; unset PGPASSWORD; /usr/pgsql-9.3/bin/pg_ctl  -D 
> /var/lib/pgsql/9.3/data/ -l /dev/null -o '\''-c 
> config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\'' start'
>  >  stderr: + 17:14:13: runasowner:467: local quietrun=
>  >  stderr: + 17:14:13: runasowner:468: local loglevel=-err
>  >  stderr: + 17:14:13: runasowner:469: local var
>  >  stderr: + 17:14:13: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:13: runasowner:473: case "$1" in
>  >  stderr: + 17:14:13: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:13: runasowner:473: case "$1" in
>  >  stderr: + 17:14:13: runasowner:485: ocf_run -err runuser postgres 
> -c 'cd /var/lib/pgsql/9.3/data/; unset PGUSER; unset PGPASSWORD; 
> /usr/pgsql-9.3/bin/pg_ctl  -D /var/lib/pgsql/9.3/data/ -l /dev/null -o 
> '\''-c config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\'' start'
>  >  stderr: + 17:14:13: ocf_run:413: local rc
>  >  stderr: + 17:14:13: ocf_run:414: local output
>  >  stderr: + 17:14:13: ocf_run:415: local verbose=1
>  >  stderr: + 17:14:13: ocf_run:416: local loglevel=err
>  >  stderr: + 17:14:13: ocf_run:417: local var
>  >  stderr: + 17:14:13: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:13: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:13: ocf_run:426: sed -e s/-//g
>  >  stderr: ++ 17:14:13: ocf_run:426: echo -err
>  >  stderr: + 17:14:13: ocf_run:426: loglevel=err
>  >  stderr: + 17:14:13: ocf_run:427: shift 1
>  >  stderr: + 17:14:13: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:13: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:13: ocf_run:433: runuser postgres -c 'cd 
> /var/lib/pgsql/9.3/data/; unset PGUSER; unset PGPASSWORD; 
> /usr/pgsql-9.3/bin/pg_ctl  -D /var/lib/pgsql/9.3/data/ -l /dev/null -o 
> '\''-c config_file=/var/lib/pgsql/9.3/data//postgresql.conf'\'' start'
>  >  stderr: + 17:14:13: ocf_run:433: output='server starting'
>  >  stderr: + 17:14:13: ocf_run:434: rc=0
>  >  stderr: ++ 17:14:13: ocf_run:435: echo server starting
>  >  stderr: + 17:14:13: ocf_run:435: output='server starting'
>  >  stderr: + 17:14:13: ocf_run:436: '[' 0 -eq 0 ']'
>  >  stderr: + 17:14:13: ocf_run:437: '[' 1 -a '!' -z 'server starting' 
> ']'
>  >  stderr: + 17:14:13: ocf_run:438: ocf_log info 'server starting'
>  >  stderr: + 17:14:13: ocf_log:321: '[' 2 -lt 2 ']'
>  >  stderr: + 17:14:13: ocf_log:325: __OCF_PRIO=info
>  >  stderr: + 17:14:13: ocf_log:326: shift
>  >  stderr: + 17:14:13: ocf_log:327: __OCF_MSG='server starting'
>  >  stderr: + 17:14:13: ocf_log:329: case "${__OCF_PRIO}" in
>  >  stderr: + 17:14:13: ocf_log:333: __OCF_PRIO=INFO
>  >  stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>  >  stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: server starting'
>  >  stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: server starting'
>  >  stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>  >  stderr: + 17:14:14: __ha_log:186: local loglevel
>  >  stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: server starting' = 
> x--ignore-stderr ']'
>  >  stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>  >  stderr: + 17:14:14: __ha_log:192: tty
>  >  stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>  >  stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>  >  stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>  >  stderr: + 17:14:14: __ha_log:202: echo 'INFO: server starting'
>  >  stderr: INFO: server starting
>  >  stderr: + 17:14:14: __ha_log:204: return 0
>  >  stderr: + 17:14:14: ocf_run:440: return 0
>  >  stderr: + 17:14:14: pgsql_real_start:593: '[' 0 -eq 0 ']'
>  >  stderr: + 17:14:14: pgsql_real_start:595: ocf_log info 'PostgreSQL 
> start command sent.'
>  >  stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>  >  stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=info
>  >  stderr: + 17:14:14: ocf_log:326: shift
>  >  stderr: + 17:14:14: ocf_log:327: __OCF_MSG='PostgreSQL start 
> command sent.'
>  >  stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>  >  stderr: + 17:14:14: ocf_log:333: __OCF_PRIO=INFO
>  >  stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>  >  stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: PostgreSQL start 
> command sent.'
>  >  stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: PostgreSQL start 
> command sent.'
>  >  stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>  >  stderr: + 17:14:14: __ha_log:186: local loglevel
>  >  stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: PostgreSQL start 
> command sent.' = x--ignore-stderr ']'
>  >  stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>  >  stderr: + 17:14:14: __ha_log:192: tty
>  >  stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>  >  stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>  >  stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>  >  stderr: + 17:14:14: __ha_log:202: echo 'INFO: PostgreSQL start 
> command sent.'
>  >  stderr: INFO: PostgreSQL start command sent.
>  >  stderr: + 17:14:14: __ha_log:204: return 0
>  >  stderr: + 17:14:14: pgsql_real_start:601: :
>  >  stderr: + 17:14:14: pgsql_real_start:603: pgsql_real_monitor warn
>  >  stderr: + 17:14:14: pgsql_real_monitor:905: local loglevel
>  >  stderr: + 17:14:14: pgsql_real_monitor:906: local rc
>  >  stderr: + 17:14:14: pgsql_real_monitor:907: local output
>  >  stderr: + 17:14:14: pgsql_real_monitor:910: loglevel=warn
>  >  stderr: + 17:14:14: pgsql_real_monitor:912: pgsql_status
>  >  stderr: + 17:14:14: pgsql_status:866: '[' -f 
> /var/lib/pgsql/9.3/data//postmaster.pid ']'
>  >  stderr: ++ 17:14:14: pgsql_status:868: head -n 1 
> /var/lib/pgsql/9.3/data//postmaster.pid
>  >  stderr: + 17:14:14: pgsql_status:868: PID=9556
>  >  stderr: + 17:14:14: pgsql_status:869: runasowner 'kill -s 0 9556 
> >/dev/null 2>&1'
>  >  stderr: + 17:14:14: runasowner:467: local quietrun=
>  >  stderr: + 17:14:14: runasowner:468: local loglevel=-err
>  >  stderr: + 17:14:14: runasowner:469: local var
>  >  stderr: + 17:14:14: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:14: runasowner:473: case "$1" in
>  >  stderr: + 17:14:14: runasowner:471: for var in 1 2
>  >  stderr: + 17:14:14: runasowner:473: case "$1" in
>  >  stderr: + 17:14:14: runasowner:485: ocf_run -err runuser postgres 
> -c 'cd /var/lib/pgsql/9.3/data/; kill -s 0 9556 >/dev/null 2>&1'
>  >  stderr: + 17:14:14: ocf_run:413: local rc
>  >  stderr: + 17:14:14: ocf_run:414: local output
>  >  stderr: + 17:14:14: ocf_run:415: local verbose=1
>  >  stderr: + 17:14:14: ocf_run:416: local loglevel=err
>  >  stderr: + 17:14:14: ocf_run:417: local var
>  >  stderr: + 17:14:14: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:14: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:14: ocf_run:426: echo -err
>  >  stderr: ++ 17:14:14: ocf_run:426: sed -e s/-//g
>  >  stderr: + 17:14:14: ocf_run:426: loglevel=err
>  >  stderr: + 17:14:14: ocf_run:427: shift 1
>  >  stderr: + 17:14:14: ocf_run:419: for var in 1 2
>  >  stderr: + 17:14:14: ocf_run:421: case "$1" in
>  >  stderr: ++ 17:14:14: ocf_run:433: runuser postgres -c 'cd 
> /var/lib/pgsql/9.3/data/; kill -s 0 9556 >/dev/null 2>&1'
>  >  stderr: + 17:14:14: ocf_run:433: output=
>  >  stderr: + 17:14:14: ocf_run:434: rc=0
>  >  stderr: ++ 17:14:14: ocf_run:435: echo
>  >  stderr: + 17:14:14: ocf_run:435: output=
>  >  stderr: + 17:14:14: ocf_run:436: '[' 0 -eq 0 ']'
>  >  stderr: + 17:14:14: ocf_run:437: '[' 1 -a '!' -z '' ']'
>  >  stderr: + 17:14:14: ocf_run:440: return 0
>  >  stderr: + 17:14:14: pgsql_status:870: return 0
>  >  stderr: + 17:14:14: pgsql_real_monitor:918: is_replication
>  >  stderr: + 17:14:14: is_replication:1287: '[' sync '!=' none -a 
> sync '!=' slave ']'
>  >  stderr: + 17:14:14: is_replication:1288: return 0
>  >  stderr: ++ 17:14:14: pgsql_real_monitor:920: exec_sql 'select 
> pg_is_in_recovery()'
>  >  stderr: ++ 17:14:14: exec_sql:518: local 'sql=select 
> pg_is_in_recovery()'
>  >  stderr: ++ 17:14:14: exec_sql:519: local output
>  >  stderr: ++ 17:14:14: exec_sql:520: local rc
>  >  stderr: +++ 17:14:14: exec_sql:524: runuser postgres -c 'cd 
> /var/lib/pgsql/9.3/data/;                 /usr/pgsql-9.3/bin/psql -p 
> 5432 -U postgres template1 -U postgres                 -Atc "select 
> pg_is_in_recovery()"'
>  >  stderr: ++ 17:14:14: exec_sql:524: output=f
>  >  stderr: ++ 17:14:14: exec_sql:525: rc=0
>  >  stderr: ++ 17:14:14: exec_sql:527: echo f
>  >  stderr: ++ 17:14:14: exec_sql:528: return 0
>  >  stderr: + 17:14:14: pgsql_real_monitor:920: output=f
>  >  stderr: + 17:14:14: pgsql_real_monitor:921: rc=0
>  >  stderr: + 17:14:14: pgsql_real_monitor:923: '[' 0 -ne 0 ']'
>  >  stderr: + 17:14:14: pgsql_real_monitor:928: case "$output" in
>  >  stderr: + 17:14:14: pgsql_real_monitor:929: ocf_log debug 
> 'PostgreSQL is running as a primary.'
>  >  stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>  >  stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=debug
>  >  stderr: + 17:14:14: ocf_log:326: shift
>  >  stderr: + 17:14:14: ocf_log:327: __OCF_MSG='PostgreSQL is running 
> as a primary.'
>  >  stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>  >  stderr: + 17:14:14: ocf_log:334: __OCF_PRIO=DEBUG
>  >  stderr: + 17:14:14: ocf_log:338: '[' DEBUG = DEBUG ']'
>  >  stderr: + 17:14:14: ocf_log:339: ha_debug 'DEBUG: PostgreSQL is 
> running as a primary.'
>  >  stderr: + 17:14:14: ha_debug:258: '[' x = x0 ']'
>  >  stderr: + 17:14:14: ha_debug:261: tty
>  >  stderr: + 17:14:14: ha_debug:262: '[' '' ']'
>  >  stderr: + 17:14:14: ha_debug:265: echo 'DEBUG: PostgreSQL is 
> running as a primary.'
>  >  stderr: DEBUG: PostgreSQL is running as a primary.
>  >  stderr: + 17:14:14: ha_debug:267: return 0
>  >  stderr: + 17:14:14: pgsql_real_monitor:930: '[' 'select now();' = 
> 'select now();' ']'
>  >  stderr: + 17:14:14: pgsql_real_monitor:931: return 8
>  >  stderr: + 17:14:14: pgsql_real_start:604: rc=8
>  >  stderr: + 17:14:14: pgsql_real_start:605: '[' 8 -eq 0 -o 8 -eq 8 ']'
>  >  stderr: + 17:14:14: pgsql_real_start:606: break
>  >  stderr: + 17:14:14: pgsql_real_start:614: use_replication_slot
>  >  stderr: + 17:14:14: use_replication_slot:1294: '[' -n '' ']'
>  >  stderr: + 17:14:14: use_replication_slot:1298: return 1
>  >  stderr: + 17:14:14: pgsql_real_start:622: ocf_log info 'PostgreSQL 
> is started.'
>  >  stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>  >  stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=info
>  >  stderr: + 17:14:14: ocf_log:326: shift
>  >  stderr: + 17:14:14: ocf_log:327: __OCF_MSG='PostgreSQL is started.'
>  >  stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>  >  stderr: + 17:14:14: ocf_log:333: __OCF_PRIO=INFO
>  >  stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>  >  stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: PostgreSQL is 
> started.'
>  >  stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: PostgreSQL is 
> started.'
>  >  stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>  >  stderr: + 17:14:14: __ha_log:186: local loglevel
>  >  stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: PostgreSQL is 
> started.' = x--ignore-stderr ']'
>  >  stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>  >  stderr: + 17:14:14: __ha_log:192: tty
>  >  stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>  >  stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>  >  stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>  >  stderr: + 17:14:14: __ha_log:202: echo 'INFO: PostgreSQL is started.'
>  >  stderr: INFO: PostgreSQL is started.
>  >  stderr: + 17:14:14: __ha_log:204: return 0
>  >  stderr: + 17:14:14: pgsql_real_start:623: return 8
>  >  stderr: + 17:14:14: pgsql_promote:690: rc=8
>  >  stderr: + 17:14:14: pgsql_promote:691: '[' 8 -ne 8 ']'
>  >  stderr: + 17:14:14: pgsql_promote:723: change_data_status pm1 LATEST
>  >  stderr: + 17:14:14: change_data_status:1569: local output
>  >  stderr: + 17:14:14: change_data_status:1571: node_exist pm1
>  >  stderr: + 17:14:14: node_exist:1720: print_crm_mon
>  >  stderr: + 17:14:14: node_exist:1720: grep -q '^node pm1'
>  >  stderr: + 17:14:14: node_exist:1720: tr '[A-Z]' '[a-z]'
>  >  stderr: + 17:14:14: print_crm_mon:1924: '[' -z '' ']'
>  >  stderr: ++ 17:14:14: print_crm_mon:1925: exec_with_retry 0 crm_mon 
> -n1
>  >  stderr: ++ 17:14:14: exec_with_retry:1689: local count=86400
>  >  stderr: ++ 17:14:14: exec_with_retry:1690: local output
>  >  stderr: ++ 17:14:14: exec_with_retry:1691: local rc
>  >  stderr: ++ 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:14: exec_with_retry:1696: shift
>  >  stderr: ++ 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>  >  stderr: +++ 17:14:14: exec_with_retry:1699: crm_mon -n1
>  >  stderr: ++ 17:14:14: exec_with_retry:1699: output='Last updated: 
> Mon Oct 12 17:14:14 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:14: exec_with_retry:1700: rc=0
>  >  stderr: ++ 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:14: exec_with_retry:1706: printf 'Last updated: 
> Mon Oct 12 17:14:14 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:14: exec_with_retry:1707: return 0
>  >  stderr: + 17:14:14: print_crm_mon:1925: CRM_MON_OUTPUT='Last 
> updated: Mon Oct 12 17:14:14 2015        Last change: Mon Oct 12 
> 16:07:56 2015 by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: + 17:14:14: print_crm_mon:1927: printf 'Last updated: Mon 
> Oct 12 17:14:14 2015        Last change: Mon Oct 12 16:07:56 2015 by 
> root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave\n'
>  >  stderr: + 17:14:14: change_data_status:1575: :
>  >  stderr: ++ 17:14:14: change_data_status:1577: 
> /usr/sbin/crm_attribute -l forever -N pm1 -n pgsql-data-status -G -q
>  >  stderr: + 17:14:14: change_data_status:1577: output=LATEST
>  >  stderr: + 17:14:14: change_data_status:1578: '[' LATEST '!=' 
> LATEST ']'
>  >  stderr: + 17:14:14: change_data_status:1582: break
>  >  stderr: + 17:14:14: change_data_status:1585: return 0
>  >  stderr: + 17:14:14: pgsql_promote:724: exec_with_retry 0 
> /usr/sbin/crm_master -l reboot -v 1000
>  >  stderr: + 17:14:14: exec_with_retry:1689: local count=86400
>  >  stderr: + 17:14:14: exec_with_retry:1690: local output
>  >  stderr: + 17:14:14: exec_with_retry:1691: local rc
>  >  stderr: + 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>  >  stderr: + 17:14:14: exec_with_retry:1696: shift
>  >  stderr: + 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>  >  stderr: ++ 17:14:14: exec_with_retry:1699: /usr/sbin/crm_master -l 
> reboot -v 1000
>  >  stderr: + 17:14:14: exec_with_retry:1699: output=
>  >  stderr: + 17:14:14: exec_with_retry:1700: rc=0
>  >  stderr: + 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>  >  stderr: + 17:14:14: exec_with_retry:1706: printf ''
>  >  stderr: + 17:14:14: exec_with_retry:1707: return 0
>  >  stderr: + 17:14:14: pgsql_promote:725: change_pgsql_status pm1 PRI
>  >  stderr: + 17:14:14: change_pgsql_status:1543: local output
>  >  stderr: + 17:14:14: change_pgsql_status:1545: is_node_online pm1
>  >  stderr: + 17:14:14: is_node_online:1716: print_crm_mon
>  >  stderr: + 17:14:14: is_node_online:1716: tr '[A-Z]' '[a-z]'
>  >  stderr: + 17:14:14: is_node_online:1716: grep -e '^node pm1 ' -e 
> '^node pm1:'
>  >  stderr: + 17:14:14: is_node_online:1716: grep -q -v offline
>  >  stderr: + 17:14:14: print_crm_mon:1924: '[' -z '' ']'
>  >  stderr: ++ 17:14:14: print_crm_mon:1925: exec_with_retry 0 crm_mon 
> -n1
>  >  stderr: ++ 17:14:14: exec_with_retry:1689: local count=86400
>  >  stderr: ++ 17:14:14: exec_with_retry:1690: local output
>  >  stderr: ++ 17:14:14: exec_with_retry:1691: local rc
>  >  stderr: ++ 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:14: exec_with_retry:1696: shift
>  >  stderr: ++ 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>  >  stderr: +++ 17:14:14: exec_with_retry:1699: crm_mon -n1
>  >  stderr: ++ 17:14:14: exec_with_retry:1699: output='Last updated: 
> Mon Oct 12 17:14:14 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:14: exec_with_retry:1700: rc=0
>  >  stderr: ++ 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>  >  stderr: ++ 17:14:14: exec_with_retry:1706: printf 'Last updated: 
> Mon Oct 12 17:14:14 2015        Last change: Mon Oct 12 16:07:56 2015 
> by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: ++ 17:14:14: exec_with_retry:1707: return 0
>  >  stderr: + 17:14:14: print_crm_mon:1925: CRM_MON_OUTPUT='Last 
> updated: Mon Oct 12 17:14:14 2015        Last change: Mon Oct 12 
> 16:07:56 2015 by root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave'
>  >  stderr: + 17:14:14: print_crm_mon:1927: printf 'Last updated: Mon 
> Oct 12 17:14:14 2015        Last change: Mon Oct 12 16:07:56 2015 by 
> root via crm_attribute on pm1
>  >  stderr: Stack: corosync
>  >  stderr: Current DC: pm2 (version 1.1.13-a14efad) - partition with 
> quorum
>  >  stderr: 2 nodes and 2 resources configured
>  >  stderr:
>  >  stderr: Node pm1: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Started
>  >  stderr: Node pm2: online
>  >  stderr:     pgsql    (ocf::heartbeat:pgsql):    Slave\n'
>  >  stderr: ++ 17:14:14: change_pgsql_status:1549: 
> /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-status -G -q
>  >  stderr: + 17:14:14: change_pgsql_status:1549: output=HS:alone
>  >  stderr: + 17:14:14: change_pgsql_status:1550: '[' HS:alone '!=' 
> PRI ']'
>  >  stderr: + 17:14:14: change_pgsql_status:1554: '[' HS:alone = STOP 
> -o HS:alone = UNKNOWN ']'
>  >  stderr: + 17:14:14: change_pgsql_status:1560: ocf_log info 
> 'Changing pgsql-status on pm1 : HS:alone->PRI.'
>  >  stderr: + 17:14:14: ocf_log:321: '[' 2 -lt 2 ']'
>  >  stderr: + 17:14:14: ocf_log:325: __OCF_PRIO=info
>  >  stderr: + 17:14:14: ocf_log:326: shift
>  >  stderr: + 17:14:14: ocf_log:327: __OCF_MSG='Changing pgsql-status 
> on pm1 : HS:alone->PRI.'
>  >  stderr: + 17:14:14: ocf_log:329: case "${__OCF_PRIO}" in
>  >  stderr: + 17:14:14: ocf_log:333: __OCF_PRIO=INFO
>  >  stderr: + 17:14:14: ocf_log:338: '[' INFO = DEBUG ']'
>  >  stderr: + 17:14:14: ocf_log:341: ha_log 'INFO: Changing 
> pgsql-status on pm1 : HS:alone->PRI.'
>  >  stderr: + 17:14:14: ha_log:253: __ha_log 'INFO: Changing 
> pgsql-status on pm1 : HS:alone->PRI.'
>  >  stderr: + 17:14:14: __ha_log:185: local ignore_stderr=false
>  >  stderr: + 17:14:14: __ha_log:186: local loglevel
>  >  stderr: + 17:14:14: __ha_log:188: '[' 'xINFO: Changing 
> pgsql-status on pm1 : HS:alone->PRI.' = x--ignore-stderr ']'
>  >  stderr: + 17:14:14: __ha_log:190: '[' none = '' ']'
>  >  stderr: + 17:14:14: __ha_log:192: tty
>  >  stderr: + 17:14:14: __ha_log:193: '[' x = x0 -a x = xdebug ']'
>  >  stderr: + 17:14:14: __ha_log:195: '[' false = true ']'
>  >  stderr: + 17:14:14: __ha_log:199: '[' '' ']'
>  >  stderr: + 17:14:14: __ha_log:202: echo 'INFO: Changing 
> pgsql-status on pm1 : HS:alone->PRI.'
>  >  stderr: INFO: Changing pgsql-status on pm1 : HS:alone->PRI.
>  >  stderr: + 17:14:14: __ha_log:204: return 0
>  >  stderr: + 17:14:14: change_pgsql_status:1561: exec_with_retry 0 
> /usr/sbin/crm_attribute -l reboot -N pm1 -n pgsql-status -v PRI
>  >  stderr: + 17:14:14: exec_with_retry:1689: local count=86400
>  >  stderr: + 17:14:14: exec_with_retry:1690: local output
>  >  stderr: + 17:14:14: exec_with_retry:1691: local rc
>  >  stderr: + 17:14:14: exec_with_retry:1693: '[' 0 -ne 0 ']'
>  >  stderr: + 17:14:14: exec_with_retry:1696: shift
>  >  stderr: + 17:14:14: exec_with_retry:1698: '[' 86400 -gt 0 ']'
>  >  stderr: ++ 17:14:14: exec_with_retry:1699: /usr/sbin/crm_attribute 
> -l reboot -N pm1 -n pgsql-status -v PRI
>  >  stderr: + 17:14:14: exec_with_retry:1699: output=
>  >  stderr: + 17:14:14: exec_with_retry:1700: rc=0
>  >  stderr: + 17:14:14: exec_with_retry:1701: '[' 0 -ne 0 ']'
>  >  stderr: + 17:14:14: exec_with_retry:1706: printf ''
>  >  stderr: + 17:14:14: exec_with_retry:1707: return 0
>  >  stderr: + 17:14:14: change_pgsql_status:1563: return 0
>  >  stderr: + 17:14:14: pgsql_promote:726: return 0
>  >  stderr: + 17:14:14: 2058: exit 0
>
>
> Thanks in advance!
>
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

-- 
Nikolay Popov
n.popov at postgrespro.ru
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company





More information about the Users mailing list