[ClusterLabs] Frequent PAF log messages - Forbidding promotion on <node> in state "startup"

[Shobe, Casey]

Thanks, I should have seen that.  I just assumed that everything was working fine because `pcs status` shows no errors.

This leads me to another question - is there a way to trigger a rebuild of a slave with pcs?  Or do I need to use `pcs cluster stop`, then manually do a new pg_basebackup, copy in the recovery.conf, and `pcs cluster start` for each standby node needing rebuilt?

> On May 13, 2018, at 5:58 AM, Jehan-Guillaume de Rorthais <jgdr at dalibo.com> wrote:
> 
> This message originated outside of DISH and was sent by: jgdr at dalibo.com
> 
> On Fri, 11 May 2018 16:25:18 +0000
> "Shobe, Casey" <Casey.Shobe at sling.com> wrote:
> 
>> I'm using PAF and my corosync log ends up filled with messages like this
>> (about 3 times per minute for each standby node):
>> 
>> pgsqlms(postgresql-10-main)[26822]:     2018/05/11_06:47:08  INFO: Forbidding
>> promotion on "d-gp2-dbp63-1" in state "startup"
>> pgsqlms(postgresql-10-main)[26822]:     2018/05/11_06:47:08  INFO: Forbidding
>> promotion on "d-gp2-dbp63-2" in state "startup"
>> 
>> What is the cause of this logging and does it indicate something is wrong
>> with my setup?
> 
> Yes, something is wrong with your setup. When a PostgreSQL standby is starting
> up, it tries to hook replication with the primary instance: this is the
> "startup" state. As soon as it is connected, it start replicating and tries to
> catchup with the master location, this is the "catchup" state. As soon as the
> standby is in sync with the master, it enters in "streaming" state. 
> See column "state" in the doc:
> https://www.postgresql.org/docs/current/static/monitoring-stats.html#PG-STAT-REPLICATION-VIEW
> 
> If you have one standby stuck in "startup" state, that means it was able to
> connect to the master but is not replicating with it for some reason
> (different/incompatible/non catchable timeline?).
> 
> Look for errors in your PostgreSQL logs on the primary and the standby.
> 
>