[ClusterLabs Developers] problem with master score limited to 1000000

[Andrew Beekhof]

> On 25 Apr 2015, at 1:33 am, Jehan-Guillaume de Rorthais <jgdr at dalibo.com> wrote:
> 
> Hi all,
> 
> We are writing a new resource agent for PostgreSQL (I am open to discuss why
> offlist to keep the thread clean) and are experiencing some limitation regarding
> to the master scoring in Pacemaker.
> 
> The only way in PostgreSQL to define which node should be promoted is to
> compare their location in their transaction log (called LSN). This LSN is
> expressed as a size that is obviously growing quickly.

We can look at bumping infinity, but what value would be acceptable?
Would using "seconds since X" be an option instead?

> Eg., After creating a
> fresh PostgerSQL instance, it is already at a few MB.
> 
> Unfortunately, we can not use this LSN as a master core because as soon as
> it is superior to 1,000,000, Pacemaker consider it internally as "infinity". In
> this case, all the slave nodes having the same score of "infinity", Pacemaker
> take a decision out of our control, probably promoting the wrong slave, leading
> to the cluster corruption from the PostgreSQL point of view.
> 
> Even if we keep track of the highest LSN during the monitor action to
> subtract it from the current LSN on pre-promote it can quickly grow over 1MB
> depending on the database activity or the monitor interval.
> 
> On pre-promote notification, we could probably :
> 
>  * abort the transition by setting -inf as master score to all the node to
>    forbid Pacemaker to promote a random slave
>  * all the node set their LSN as an attribute called "lsn" as instance
> 
> Each resource agent would then be able to compare their LSN to each others and
> set a master if it has the best one.
> 
> But then, we are not sure how to trigger this second round of the election.
> Waiting for a monitor action to trigger on all the nodes sounds like an
> eternity to wait. Moreover, this sounds like the wrong place to call crm_master
> as we don't know for sure all the other node had a monitor call before the
> promotion is triggered.
> 
> Any idea or advice, comments ? Thanks in advance !
> 
> Regards,
> -- 
> Jehan-Guillaume de Rorthais
> Dalibo
> http://www.dalibo.com
> 
> _______________________________________________
> Developers mailing list
> Developers at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/developers