[Pacemaker] Custom event script during master/slave failover?

[David Vossel]

----- Original Message -----
> From: "Cal Heldenbrand" <cal at fbsdata.com>
> To: "The Pacemaker cluster resource manager" <pacemaker at oss.clusterlabs.org>
> Sent: Monday, November 12, 2012 3:27:02 PM
> Subject: Re: [Pacemaker] Custom event script during master/slave failover?
> 
> 
> I've managed to get a test Redis configuration up and running...
> however this is the first time I've played with a master/slave
> resource, and I'm noticing some odd behavior in the monitoring
> actions. Setting in a super quick monitoring interval of 1 second, I
> noticed at first only the slave was being monitored, 2x in each
> second. Then after changing a bunch of config options and several
> cluster restarts, I noticed the monitoring action had moved to the
> master, again 2x events in each second.

Use a longer timeout than 1s. Don't go under 20s... that might be why you see 2x events.

The monitor action didn't move to the master or move to the slave.  There is a separate monitor operation used for resources in the master and slave role.  Both monitor operations have to be defined, looking at your config it appears only the Master monitor op is defined.

http://www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/_monitoring_multi_state_resources.html

If you are new to multi-state resources. I'd recommend playing around with the ocf:pacemaker:Stateful resource some.  Stateful is a fake resource that does nothing except allow you to start/stop/promote/demote/monitor it.  It is useful for testing out configurations and understanding expected behavior.

-- Vossel



> 
> redis(redis:1)[2203]: 2012/11/12_15:17:52 INFO: redis-server is
> running (PID 30759)
> redis(redis:1)[2203]: 2012/11/12_15:17:52 INFO: redis-server is
> running (PID 30759)
> redis(redis:1)[2203]: 2012/11/12_15:17:52 INFO: redis-server is
> running (as master)
> redis(redis:1)[2203]: 2012/11/12_15:17:52 INFO: redis-server is
> running (as master)
> 
> Here is my primitive and master/slave definition:
> 
> primitive redis ocf:heartbeat:redis \
> meta target-role="Master" is-managed="true" \
> op monitor interval="1s" role="Master" timeout="5s" on-fail="restart"
> ms redis_clone redis \
> meta notify="true" is-managed="true" ordered="false"
> interleave="false" globally-unique="false" target-role="Master"
> migration-threshold="1"
> 
> I'd like to figure out how to make sure both master and slave are
> actively monitored. Let me know if you need any more info on my
> config!
> 
> Thank you,
> 
> --Cal
> 
> 
> 
> On Fri, Nov 9, 2012 at 4:52 PM, David Vossel < dvossel at redhat.com >
> wrote:
> 
> 
> 
> 
> 
> ----- Original Message -----
> > From: "Cal Heldenbrand" < cal at fbsdata.com >
> > To: "The Pacemaker cluster resource manager" <
> > pacemaker at oss.clusterlabs.org >
> > Sent: Friday, November 9, 2012 4:29:05 PM
> > Subject: [Pacemaker] Custom event script during master/slave
> > failover?
> > 
> > 
> > Hi everyone,
> > 
> > I'm playing around with the possibility of using Pacemaker in a
> > Redis
> > master/slave cluster. The difficult part of this, is that Redis
> > will
> > not automatically flip itself from read-only slave mode into master
> > mode. A client needs to connect to the slave server and run this
> > SLAVEOF NO ONE command to turn it into a master.
> 
> The resource agent should be able to do this. Pacemaker will tell the
> resource agent to promote/demote a resource. Your Redis agent can
> then go off and do whatever logic is necessary to actually
> transition the process a master/slave process.
> 
> 
> > 
> > If I set up a master/slave primitive, is it possible to fire an
> > event
> > script during a failover event that would allow me to do this?
> > 
> > After the stonith event fires and the broken master reboots,
> > I would
> > write in something special in the Redis init.d script to detect if
> > my peer is already a master, and start up in slave mode.
> 
> If you are using Pacemaker, it should be controlling what node has
> the master resource and what node has the slave resource. Take a
> look at what OCF scripts are and how pacemaker uses them.
> 
> 
> > 
> > Has anyone else worked up a cluster config with Redis?
> 
> Using google, found some work someone did on a redis resource agent.
> https://github.com/martinwalter/resource-agents/blob/master/heartbeat/redis
> 
> -- Vossel
> > Thank you,
> > 
> > --Cal
> > 
> > _______________________________________________
> > Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> > 
> > Project Home: http://www.clusterlabs.org
> > Getting started:
> > http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> > Bugs: http://bugs.clusterlabs.org
> > 
> 
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> 
> Project Home: http://www.clusterlabs.org
> Getting started:
> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
> 
> 
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> 
> Project Home: http://www.clusterlabs.org
> Getting started:
> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>