[Pacemaker] custom resource-agent

Tue Oct 1 23:54:07 EDT 2013

 yes i need to use the aws command, i am using a VPC, after issueing the
command i get a "true" statement and its done

so i only want pacemaker to issue the one-shot command at failover. Here is
what i have atm: (i know its still dirty, just lerning pacemaker)

primitive drbd_mysql ocf:linbit:drbd \
        params drbd_resource="mydata" \
        op monitor interval="15s"
primitive fs_mysql ocf:heartbeat:Filesystem \
        params device="/dev/drbdx" directory="/mountpint" fstype="ext4"
options="relatime,barrier=1" \
        op start interval="0" timeout="60" \
        op stop interval="0" timeout="60" \
        op monitor interval="10s" timeout="60s" OCF_CHECK_LEVEL="20" \
        meta target-role="started"
primitive fvip ocf:heartbeat:AWSFIP \
        params fvip="192.168.2.10" region="ap-southeast-1"
primitive ip_mysql ocf:heartbeat:IPaddr2 \
        params ip="192.168.2.10" cidr_netmask="20" \
        op monitor interval="10" \
        meta target-role="started"
primitive mysqld lsb:mysql
group mysql fs_mysql ip_mysql mysqld
ms ms_drbd_mysql drbd_mysql \
        meta master-max="1" master-node-max="1" clone-max="2"
clone-node-max="1" notify="true"
colocation mysql_on_drbd inf: fvip mysql ms_drbd_mysql:Master
order mysql_after_drbd_and_fvip inf: ms_drbd_mysql:promote fvip:start
mysql:start

my AWSFIP(adjusted Dummy :%s/dummy/awsfip/g|%s/Dummy/AWSFIP/g):

<parameter name="fvip" unique="1" required="1">
<longdesc lang="en">
The IPv4 address to be configured in dotted quad notation, for example
"192.168.1.1".
</longdesc>
<shortdesc lang="en">IPv4 address</shortdesc>
<content type="string" default="" />
</parameter>

<parameter name="region" unique="1" required="1">
<longdesc lang="en">
The name of the AWS region
</longdesc>
<shortdesc lang="en">AWS region</shortdesc>
<content type="string"/>
</parameter>

awsfip_start() {
    awsfip_monitor
    Instance_ID=`/usr/bin/curl --silent
http://169.254.169.254/latest/meta-data/instance-id`
    ENI_ID=`aws ec2 describe-instances --instance-id $Instance_ID --region
$OCF_RESKEY_region | grep NetworkInterfaceId | cut -d '"' -f 4`
    if [ $? =  $OCF_SUCCESS ]; then
        return $OCF_SUCCESS
    fi
         aws ec2 assign-private-ip-addresses --network-interface-id $ENI_ID
--private-ip-addresses $OCF_RESKEY_fvip --allow-reassignment --region
$OCF_RESKEY_region
         sleep 4
         aws ec2 assign-private-ip-addresses --network-interface-id $ENI_ID
--private-ip-addresses $OCF_RESKEY_fvip --allow-reassignment --region
$OCF_RESKEY_region
         /etc/init.d/networking restart
    touch ${OCF_RESKEY_state}
}

I couldn't get it to work yet, and i don't want to run a external script
for it. I can't be so hard to let pacemaker execute an additional one-shot
command at failover (in the correct order..)

Thanks for your help!

On Wednesday, October 02, 2013 07:33 AM, David Lang wrote:

the aws command is making the call to inform aws, if you were to bring up
the address without making the aws command, would it work? If you are on a
Virtual Private Cloud (VPC), it may, but I didn't think it would.

If you can make it work without the aws command, then you can just use the
standard pacemaker VIP configuration. I know that this doesn't work if you
have an external IP that you are moving (you must use an aws call to tell
Amazon to move the IP), but it's possible that you don't have to for an
internal IP, but I would be surprised.

David Lang

On Wed, 2 Oct 2013, Peter Romfeld wrote:

Hey,
when i change the secondary IP per hand or with external script on a Ubuntu
Instance I just need:
/etc/network/interfaces
auto eth0
iface eth0 inet dhcp
   address 192.168.32.12
   netmask 255.255.240.0
   gateway 192.168.32.1
   up ip addr add 192.168.32.11/20 dev eth0

and then run the script which basically just does:
#!/bin/sh

VIP=172.32.32.11
REGION=ap-southeast-1

Instance_ID=`/usr/bin/curl --silent
http://169.254.169.254/latest/meta-data/instance-id`
ENI_ID=`aws ec2 describe-instances --instance-id $Instance_ID --region
$REGION | grep NetworkInterfaceId | cut -d '"' -f 4`

aws ec2 assign-private-ip-addresses --network-interface-id $ENI_ID
--private-ip-addresses $VIP --allow-reassignment --region $REGION

I dont need to inform AWS or restart network, only the correct network
config and the one command, when i tested it with pinging from a 3rd
instance during IP change i didnt got any interupts. I dont know about
monitoring it

On Wed, Oct 2, 2013 at 1:38 AM, David Lang <david at lang.hm>
<david at lang.hm>wrote:

On Tue, 1 Oct 2013, Dejan Muhamedagic wrote:

 On Tue, Oct 01, 2013 at 10:07:12AM -0700, David Lang wrote:

On Tue, 1 Oct 2013, Dejan Muhamedagic wrote:

 On Tue, Oct 01, 2013 at 07:22:20AM -0700, David Lang wrote:

On Tue, 1 Oct 2013, Dejan Muhamedagic wrote:

 Hi David,

On Mon, Sep 30, 2013 at 12:41:23PM -0700, David Lang wrote:

On Mon, 30 Sep 2013, David Lang wrote:

 On Mon, 30 Sep 2013, Michael Schwartzkopff wrote:

 Am Montag, 30. September 2013, 21:12:56 schrieb Peter Romfeld:

I am working in AWS i cant just use a VIP i need to use a floating
secondary IP which i reassign through script, i want to let
pacemaker
handle the reassignment...

Please explain the difference of a VIP and a "secondary IP" in
your opinion.

with AWS you need to inform amazon of the change, not just change
the IP on the local box, that requires much more work than a
simple local VIP

being more detailed, instead of just
ifconfig eth0:0 $vip
you have to do something like

/opt/aws/bin/ec2-assign-**private-ip-addresses -n $ENI_ID
--secondary-private-ip-address $VIP --allow-reassignment --region $REGION

We may consider adding such an option to IPaddr2. Has anybody
ever tried that?

 pingresult=`ping -c 1 -W 1 $VIP | grep time= | wc -l`

if [ "$pingresult" == "0" ]; then
echo `date` "-- Restarting network"
/sbin/service network restart > /dev/null 2>&1

That may break the cluster communication, which may lead to split
brain, etc. Is that really the only way?

It's not the only way, but you do have the problem that the call to
aws management interface is asynchronous, you don't know when it's
going to complete, and until it does, the IP doesn't actually work.

Wouldn't it be then safer to wait until it starts working, i.e.
to monitor in a loop?

that's exactly what the snippet of code above is for, to detect when
the other box no longer has the address.

Hmm, perhaps I'm missing something, but I couldn't notice a loop
in that code. What I meant was something like this:

while ! ping -c 1 -W 1 $VIP | grep -qs time=; do
        :
done

Then network restart wouldn't be necessary, right? Sorry, I don't
know much about aws.

I haven't used this exact script before, but I have seen the problem that
this script is designed to address. I am not saying that I agree with this
script, but it's what Amazon is suggesting, so it's probably a reasonable
start.

this was a cut-n-paste from the URL provided earlier
http://aws.amazon.com/**articles/2127188135977316
<http://aws.amazon.com/articles/2127188135977316><http://aws.amazon.com/articles/2127188135977316>

#!/bin/sh
# This script will monitor another HA node and take over a Virtual IP (VIP)
# if communication with the other node fails

# High Availability IP variables
# Other node's IP to ping and VIP to swap if other node goes down
HA_Node_IP=10.0.0.11
VIP=10.0.0.10

# Specify the EC2 region that this will be running in
REGION=us-west-2

# Run aws-apitools-common.sh to set up default environment variables and to
# leverage AWS security credentials provided by EC2 roles
. /etc/profile.d/aws-apitools-**common.sh

# Determine the instance and ENI IDs so we can reassign the VIP to the
# correct ENI. Requires EC2 describe-instances and
assign-private-ip-address
# permissions. The following example EC2 roles policy will authorize these
# commands:
# {
# "Statement": [
# {
# "Action": [
# "ec2:AssignPrivateIpAddresses"**,
# "ec2:DescribeInstances"
# ],
# "Effect": "Allow",
# "Resource": "*"
# }
# ]
# }

Instance_ID=`/usr/bin/curl --silent http://169.254.169.254/latest/**
meta-data/instance-id`ENI_ID=`**/opt/aws/bin/ec2-describe-**instances
<http://169.254.169.254/latest/meta-data/instance-idENI_ID=/opt/aws/bin/ec2-describe-instances><http://169.254.169.254/latest/meta-data/instance-idENI_ID=/opt/aws/bin/ec2-describe-instances>$Instance_ID
--region $REGION | grep eni -m 1 | awk '{print $2;}'`

echo `date` "-- Starting HA monitor"
while [ . ]; do
 pingresult=`ping -c 3 -W 1 $HA_Node_IP | grep time= | wc -l`

 if [ "$pingresult" == "0" ]; then
 echo `date` "-- HA heartbeat failed, taking over VIP"

 /opt/aws/bin/ec2-assign-**private-ip-addresses -n $ENI_ID
--secondary-private-ip-address $VIP --allow-reassignment --region $REGION
 pingresult=`ping -c 1 -W 1 $VIP | grep time= | wc -l`
 if [ "$pingresult" == "0" ]; then
 echo `date` "-- Restarting network"
 /sbin/service network restart > /dev/null 2>&1
 fi
 sleep 60
 fi
 sleep 2
done

David Lang

______________________________**_________________
Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
http://oss.clusterlabs.org/**mailman/listinfo/pacemaker
<http://oss.clusterlabs.org/mailman/listinfo/pacemaker><http://oss.clusterlabs.org/mailman/listinfo/pacemaker>

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/**doc/Cluster_from_Scratch.pdf
<http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf><http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf>
Bugs: http://bugs.clusterlabs.org

_______________________________________________

Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org

Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf

Bugs: http://bugs.clusterlabs.org

_______________________________________________
Pacemaker mailing list:
Pacemaker at oss.clusterlabs.orghttp://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/pacemaker/attachments/20131002/4a3b0e08/attachment-0003.html>