<div dir="ltr"><div><div><div><div><div><div><div><div>Hi Takatoshi,<br><br></div>Thank you for your reply.<br></div>I did read the page you mentionned. Actually this is the page I use to setup such clusters (except I don't use only one VIP).<br></div>I can have initial sync working following this page. The problem I have is that, once the cluster is up (one master PRI and one slave STREAMING|SYNC), I standby the master node in order to test failover.<br></div>I would then expect the former slave to be promoted but this never seems to happen.<br></div>When I issue the crm node standby on the first master, I get the logs bellow on the slave to be promoted:<br><br>==> /var/lib/pgsql/9.1/data/pg_log/postgresql-Mon.log <==<br>FATAL: r?plication termin?e par le serveur primaire (french logs meaning: replication ended by master)<br><br>==> /var/log/messages <==<br>Mar 2 10:04:37 pp-obm-sgbd2 crmd[19626]: notice: run_graph: Transition 995 (Complete=4, Pending=0, Fired=0, Skipped=9, Incomplete=2, Source=/var/lib/pacemaker/pengine/pe-input-76.bz2): Stopped<br>Mar 2 10:04:37 pp-obm-sgbd2 pengine[19625]: notice: unpack_config: On loss of CCM Quorum: Ignore<br>Mar 2 10:04:37 pp-obm-sgbd2 pengine[19625]: notice: LogActions: <b>Promote pri_pgsql:0#011(Slave -> Master <a href="http://pp-obm-sgbd2.upond.fr">pp-obm-sgbd2.upond.fr</a>)</b><br>Mar 2 10:04:37 pp-obm-sgbd2 pengine[19625]: notice: LogActions: <b>Stop pri_pgsql:1#011(<a href="http://pp-obm-sgbd.upond.fr">pp-obm-sgbd.upond.fr</a>)</b><br>Mar 2 10:04:37 pp-obm-sgbd2 pengine[19625]: notice: LogActions: Move pri_vip#011(Started <a href="http://pp-obm-sgbd.upond.fr">pp-obm-sgbd.upond.fr</a> -> <a href="http://pp-obm-sgbd2.upond.fr">pp-obm-sgbd2.upond.fr</a>)<br>Mar 2 10:04:37 pp-obm-sgbd2 crmd[19626]: notice: te_rsc_command: Initiating action 20: stop pri_vip_stop_0 on <a href="http://pp-obm-sgbd.upond.fr">pp-obm-sgbd.upond.fr</a><br>Mar 2 10:04:37 pp-obm-sgbd2 crmd[19626]: notice: te_rsc_command: Initiating action 11: stop pri_pgsql_stop_0 on <a href="http://pp-obm-sgbd.upond.fr">pp-obm-sgbd.upond.fr</a><br>Mar 2 10:04:37 pp-obm-sgbd2 pengine[19625]: notice: process_pe_message: Calculated Transition 996: /var/lib/pacemaker/pengine/pe-input-77.bz2<br>Mar 2 10:04:37 pp-obm-sgbd2 crmd[19626]: notice: abort_transition_graph: Transition aborted by deletion of nvpair[@id='status-pp-obm-sgbd.upond.fr-pri_pgsql-xlog-loc']: Transient attribute change (cib=2.61.6, source=te_update_diff:391, path=/cib/status/node_state[@id='<a href="http://pp-obm-sgbd.upond.fr">pp-obm-sgbd.upond.fr</a>']/transient_attributes[@id='<a href="http://pp-obm-sgbd.upond.fr">pp-obm-sgbd.upond.fr</a>']/instance_attributes[@id='<a href="http://status-pp-obm-sgbd.upond.fr">status-pp-obm-sgbd.upond.fr</a>']/nvpair[@id='status-pp-obm-sgbd.upond.fr-pri_pgsql-xlog-loc'], 0)<br>Mar 2 10:04:38 pp-obm-sgbd2 crmd[19626]: notice: run_graph: Transition 996 (Complete=4, Pending=0, Fired=0, Skipped=6, Incomplete=1, Source=/var/lib/pacemaker/pengine/pe-input-77.bz2): Stopped<br>Mar 2 10:04:38 pp-obm-sgbd2 pengine[19625]: notice: unpack_config: On loss of CCM Quorum: Ignore<br>Mar 2 10:04:38 pp-obm-sgbd2 pengine[19625]: notice: LogActions: <b>Promote pri_pgsql:0#011(Slave -> Master <a href="http://pp-obm-sgbd2.upond.fr">pp-obm-sgbd2.upond.fr</a>)</b><br>Mar 2 10:04:38 pp-obm-sgbd2 pengine[19625]: notice: LogActions: Start pri_vip#011(<a href="http://pp-obm-sgbd2.upond.fr">pp-obm-sgbd2.upond.fr</a>)<br>Mar 2 10:04:38 pp-obm-sgbd2 crmd[19626]: warning: run_graph: Transition 997 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=6, Source=/var/lib/pacemaker/pengine/pe-input-78.bz2): Terminated<br>Mar 2 10:04:38 pp-obm-sgbd2 crmd[19626]: warning: te_graph_trigger: Transition failed: terminated<br>Mar 2 10:04:38 pp-obm-sgbd2 crmd[19626]: notice: print_graph: Graph 997 with 6 actions: batch-limit=6 jobs, network-delay=0ms<br>Mar 2 10:04:38 pp-obm-sgbd2 crmd[19626]: notice: print_synapse: [Action 9]: Pending rsc op pri_pgsql_monitor_15000 on <a href="http://pp-obm-sgbd2.upond.fr">pp-obm-sgbd2.upond.fr</a> (priority: 0, waiting: 8)<br>Mar 2 10:04:38 pp-obm-sgbd2 crmd[19626]: notice: print_synapse: [Action 8]: <b>Pending rsc op pri_pgsql_promote_0 on <a href="http://pp-obm-sgbd2.upond.fr">pp-obm-sgbd2.upond.fr</a> (priority: 0, waiting: 14)</b><br>Mar 2 10:04:38 pp-obm-sgbd2 crmd[19626]: notice: print_synapse: [Action 15]:<b> Pending pseudo op ms_pgsql_promoted_0 on N/A (priority: 1000000, waiting: 8)</b><br>Mar 2 10:04:38 pp-obm-sgbd2 crmd[19626]: notice: print_synapse: [Action 14]:<b> Pending pseudo op ms_pgsql_promote_0 on N/A (priority: 0, waiting: 18)</b><br>Mar 2 10:04:38 pp-obm-sgbd2 crmd[19626]: notice: print_synapse: [Action 19]: Pending rsc op pri_vip_monitor_10000 on <a href="http://pp-obm-sgbd2.upond.fr">pp-obm-sgbd2.upond.fr</a> (priority: 0, waiting: 18)<br>Mar 2 10:04:38 pp-obm-sgbd2 crmd[19626]: notice: print_synapse: [Action 18]: Pending rsc op pri_vip_start_0 on <a href="http://pp-obm-sgbd2.upond.fr">pp-obm-sgbd2.upond.fr</a> (priority: 0, waiting: 15)<br>Mar 2 10:04:38 pp-obm-sgbd2 crmd[19626]: notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]<br>Mar 2 10:04:38 pp-obm-sgbd2 pengine[19625]: notice: process_pe_message: Calculated Transition 997: /var/lib/pacemaker/pengine/pe-input-78.bz2<br><br>!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!<br>Pacemaker seems to be trying to acheive promotion of the former slave as expected, but I don't understand why crmd logs "ms_pgsql_promoted_0" and "ms_pgsql_promote_0" on "N/A"<br>!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!<br><br>==> /var/lib/pgsql/9.1/data/pg_log/postgresql-Mon.log <==<br>cp: impossible d'?valuer << /var/lib/pgsql/replication/00000002000000000000002F >>: Aucun fichier ou dossier de ce type<br>LOG: enregistrement de longueur nulle ? 0/2F000078<br>cp: impossible d'?valuer << /var/lib/pgsql/replication/00000002000000000000002F >>: Aucun fichier ou dossier de ce type<br>cp: impossible d'?valuer << /var/lib/pgsql/replication/00000003.history >>: Aucun fichier ou dossier de ce type<br>FATAL: n'a pas pu se connecter au serveur principal : n'a pas pu se connecter au serveur : Connexion termin?e par expiration du d?lai d'attente<br> Le serveur est-il actif sur l'h?te << 193.50.151.200 >> et accepte-t-il les connexions<br> TCP/IP sur le port 5432 ?<br> <br>cp: impossible d'?valuer << /var/lib/pgsql/replication/00000002000000000000002F >>: Aucun fichier ou dossier de ce type<br>cp: impossible d'?valuer << /var/lib/pgsql/replication/00000002000000000000002F >>: Aucun fichier ou dossier de ce type<br>cp: impossible d'?valuer << /var/lib/pgsql/replication/00000003.history >>: Aucun fichier ou dossier de ce type<br>FATAL: n'a pas pu se connecter au serveur principal : n'a pas pu se connecter au serveur : Aucun chemin d'acc?s pour atteindre l'h?te cible<br> Le serveur est-il actif sur l'h?te << 193.50.151.200 >> et accepte-t-il les connexions<br><br>!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!<br></div>Now the RA seems to be trying to make the node a slave again.... but htere is no master. The node stays in the state shown bellow for ever:<br><br>* Node <a href="http://pp-obm-sgbd2.upond.fr">pp-obm-sgbd2.upond.fr</a>:<br> + master-pri_pgsql : 100<br> + pri_pgsql-data-status : STREAMING|SYNC <br> + pri_pgsql-status : HS:sync <br>!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!<br><br></div>That the very fist time I see this behaviour and I don't understand what I did differents from other setups. Any advice or help to find more informations or debug traces are very welcome!<br><br><br></div>Thanks<br><div><div><br><div><br></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">2015-03-01 14:06 GMT+01:00 Takatoshi MATSUO <span dir="ltr"><<a href="mailto:matsuo.tak@gmail.com" target="_blank">matsuo.tak@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Alexandre<br>
<span class=""><br>
> pgsql(pri_pgsql)[13223]: WARNING: My data is out-of-date. status=DISCONNECT<br>
<br>
</span>Did you read Q&A ?<br>
<a href="http://clusterlabs.org/wiki/PgSQL_Replicated_Cluster" target="_blank">http://clusterlabs.org/wiki/PgSQL_Replicated_Cluster</a><br>
<br>
-------<br>
How do I force start Master although pgsql-data-status is "DISCONNECT"?<br>
# crm_attribute -l forever -N {Node Name} -n "pgsql-data-status" -v "LATEST"<br>
------<br>
<br>
Regards,<br>
Takatoshi MATSUO<br>
<div class="HOEnZb"><div class="h5"><br>
_______________________________________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org">Users@clusterlabs.org</a><br>
<a href="http://clusterlabs.org/mailman/listinfo/users" target="_blank">http://clusterlabs.org/mailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" target="_blank">http://bugs.clusterlabs.org</a><br>
</div></div></blockquote></div><br></div>