[Pacemaker] pacemaker won't start mysql in the second node

Brian Cavanagh brian at designedtoscale.com
Wed Feb 2 14:28:08 EST 2011 

NP, glad I could help, it also heartening to know that I should try a
fresh start with heartbeat to get ha to work right since you have it
failing over correctly.

Best

On 2/2/11 2:17 PM, "Liang.Ma at asc-csa.gc.ca" <Liang.Ma at asc-csa.gc.ca> wrote:

>Brian,
>
>Thanks again for your comments. I will check the mysql configuration and
>try to make them consistent.
>
>I am using heartbeat not openais/corosync.
>
>
>
>Liang Ma
>Contractuel | Consultant | SED Systems Inc.
>Ground Systems Analyst
>Agence spatiale canadienne | Canadian Space Agency
>6767, Route de l'Aéroport, Longueuil (St-Hubert), QC, Canada, J3Y 8Y9
>Tél/Tel : (450) 926-5099 | Téléc/Fax: (450) 926-5083
>Courriel/E-mail : [liang.ma at space.gc.ca]
>Site web/Web site : [www.space.gc.ca ]
>
>
>
>
>-----Original Message-----
>From: Brian Cavanagh [mailto:brian at designedtoscale.com]
>Sent: February 2, 2011 1:59 PM
>To: The Pacemaker cluster resource manager
>Subject: Re: [Pacemaker] pacemaker won't start mysql in the second node
>
>Glad to hear your failover is working fine.
>
>Mysql will work fine with out a socket file, just stake it out of your
>cib.xml and you my.cnf  you connect via localhost or something else.  I
>never use it myself.
>
>About the logs, check your permissions on them, just make sure they exist,
>and are set to 666. Have your my.cnf agree with settings in pacemaker.
>Also Check your status settings via show global settings like "%log%" \G
>make sure everything is being carried over right and that the settings are
>correct.
>I just noticed your log setting in the cib.xml should be the
>mysql_safe.log.  If the log isn't being written to the /var/log/ location
>it is most likely being written as [server host].err in /var/lib/mysql
>
>I think your prior problem has to do with setting up mysql on one machine
>the normal way, so all the directories and files were created properly,
>then you tried to failover to machine that was not set up that way.  The
>other point is that I think /var/run is wiped on reboot.  So
>/var/run/mysqld | and pid | sock were attempted to be created as service
>outside of the context of root.
>
>BTW Are you using heartbeat or openais/corosync?
>
>
>On 2/2/11 1:30 PM, "Liang.Ma at asc-csa.gc.ca" <Liang.Ma at asc-csa.gc.ca>
>wrote:
>
>>I ran the test to put active node running mysql to standby, it fails over
>>to another node nicely now. Because my problem was mysql couldn't fail
>>over to the second node while it runs fine in the first node.
>>
>>ps ax | grep mysql shows it did take /var/lib/mysql/mysqld.pid from the
>>crm configure, but didn't show any log directories. It doesn't explain
>>why my mysql-server doesn't write logs to /var/log/mysql.log and even
>>doesn't have file /var/log/mysql_safe.log.
>>
>>Or by the way, you mentioned "stop using the sock", how can I do it? Can
>>we still access mysql service without socket open?
>>
>>Best regards,
>>
>>
>>Liang Ma
>>Contractuel | Consultant | SED Systems Inc.
>>Ground Systems Analyst
>>Agence spatiale canadienne | Canadian Space Agency
>>6767, Route de l'Aéroport, Longueuil (St-Hubert), QC, Canada, J3Y 8Y9
>>Tél/Tel : (450) 926-5099 | Téléc/Fax: (450) 926-5083
>>Courriel/E-mail : [liang.ma at space.gc.ca]
>>Site web/Web site : [www.space.gc.ca ]
>>
>>
>>
>>
>>-----Original Message-----
>>From: Brian Cavanagh [mailto:brian at designedtoscale.com]
>>Sent: February 2, 2011 1:11 PM
>>To: The Pacemaker cluster resource manager
>>Subject: Re: [Pacemaker] pacemaker won't start mysql in the second node
>>
>>1) Well you had a few problems, mkdir in var/run the log file permissions
>>and the pid + sock permissions issue. When you manage services this way,
>>to stay clear of permission headaches and context switches, store the
>>pids/socks and the rest of the temporary resources in a directory owned
>>by
>>the service in question. Saves you the trouble of trying to work it out
>>post facto. 
>>
>>2) run ps ax | grep mysql
>>You will see that you were passing cli arguments which override the
>>defaults set in the conf file.
>>
>>Try running an additional test.  Put the node the mysql resource is
>>running on in standby mode, and see if it fails over correctly while
>>watching crm_mon.  Can you let me know what happens?  I myself have been
>>an issue where I can get everything running, and I can manage and migrate
>>mysql but not failover. I think it might be an issue with heartbeat, but
>>I
>>am not sure.
>>
>>Best
>>
>>On 2/2/11 12:49 PM, "Liang.Ma at asc-csa.gc.ca" <Liang.Ma at asc-csa.gc.ca>
>>wrote:
>>
>>>YES, this resolved my problem.
>>>
>>>Thank you Brian.
>>>
>>>Actually I followed your suggestion by changing the log file permission
>>>and moving the pid file from /var/run/mysqld to /var/run. It didn't
>>>change the result. Then finally I moved the pid file to /var/lib/mysql,
>>>which fixed the problem.
>>>
>>>Still two questions:
>>>1. why did it work before I upgraded mysql-server? I thought it was just
>>>a minor upgrade.
>>>2. mysql only writes logs to /var/log/mysql/error.log, which is setup by
>>>my.cnf by default. /var/log/mysql.log is always empty and
>>>/var/log/mysql_safe.log even not exists.
>>>
>>>Thank you Brian again, I will post the solution to linux-ha list.
>>>
>>>Liang Ma
>>>Contractuel | Consultant | SED Systems Inc.
>>>Ground Systems Analyst
>>>Agence spatiale canadienne | Canadian Space Agency
>>>6767, Route de l'Aéroport, Longueuil (St-Hubert), QC, Canada, J3Y 8Y9
>>>Tél/Tel : (450) 926-5099 | Téléc/Fax: (450) 926-5083
>>>Courriel/E-mail : [liang.ma at space.gc.ca]
>>>Site web/Web site : [www.space.gc.ca ]
>>>
>>>
>>>
>>>
>>>-----Original Message-----
>>>From: Brian Cavanagh [mailto:brian at designedtoscale.com]
>>>Sent: February 2, 2011 10:51 AM
>>>To: The Pacemaker cluster resource manager
>>>Subject: Re: [Pacemaker] pacemaker won't start mysql in the second node
>>>
>>>sudo bash
>>>chmod 666 /var/log/mysql_safe.log
>>>chmod 666 /var/log/mysql.log
>>>
>>>crm configure edit
>>>....
>>>primitive mysql ocf:heartbeat:mysql \
>>>    params binary="/usr/bin/mysqld_safe" config="/etc/mysql/my.cnf" \
>>>user="mysql" group="mysql" log="/var/log/mysql.log" \
>>>pid="/var/run/mysqld.pid" datadir="/var/lib/mysql" \
>>>socket="/var/run/mysqld/mysqld.sock"
>>>
>>>...
>>>
>>>
>>>That should fix your issues, if not, change your pid location to
>>>/var/lib/mysql and stop using the sock.
>>>
>>>Also tail the mysql_safe.log next time.
>>>
>>>
>>>
>>>On 2/2/11 10:05 AM, "Liang.Ma at asc-csa.gc.ca" <Liang.Ma at asc-csa.gc.ca>
>>>wrote:
>>>
>>>>Hi, 
>>>>
>>>>I did what Michael suggested (included below). When there are only
>>>>ms_drbd_mysql and fs_mysql, no problem to fail over to node 2. Added
>>>>ip1,
>>>>it still fail over to arsvr2 fine when I put node 1 (arsvr1) standby.
>>>>But
>>>>when I added mysql in group MySQLDB, it behaved exactly the same.
>>>>fs_mysql started and mounted on arsvr2, and even ip1 started no
>>>>problem,
>>>>but mysql failed to start. Crm_mon shows the error
>>>>
>>>>Failed actions: mysql_start_0 (node=arsvr2, call=32, rc=4,
>>>>status=complete): insufficient privileges
>>>>
>>>>While the cluster log didn't show anything on mysql start.
>>>>
>>>>Also I tried to run resource script mysql under
>>>>/usr/lib/ocf/resource.d/heartbeat, it started mysql server no problem.
>>>>
>>>>My guess is the problem is somewhere right before pacemaker calls
>>>>resource mysql. Maybe related to any permission or authentication
>>>>problem
>>>>with mysql as commented by Dejan, but I checked the permission on
>>>>/var/run/mysqld and /var/lib/mysql, they are the same on both nodes.
>>>>Anything within /var/lib/mysql is shared by drbd partition, which
>>>>should
>>>>be identical, right?
>>>>
>>>>Anyone has any ideas which part of mysql server setting's may cause the
>>>>problem? My.cnf files under /etc/ in both nodes are identical. I found
>>>>debian.cnf files were different in password field after upgrading. Then
>>>>I
>>>>copied the one from arsvr1 to arsvr2.
>>>>
>>>>Thank you for your any help.
>>>>
>>>>Here is the simplified crm configuration.
>>>>
>>>>
>>>>node $id="bc6bf61d-6b5f-4307-85f3-bf7bb11531bb" arsvr2 \
>>>>    attributes standby="off"
>>>>node $id="bf0e7394-9684-42b9-893b-5a9a6ecddd7e" arsvr1 \
>>>>    attributes standby="off"
>>>>primitive drbd_mysql ocf:linbit:drbd \
>>>>    params drbd_resource="r0" \
>>>>    op monitor interval="15s"
>>>>primitive fs_mysql ocf:heartbeat:Filesystem \
>>>>    params device="/dev/drbd/by-res/r0" directory="/var/lib/mysql"
>>>>fstype="ext4" \
>>>>    op start interval="0" timeout="60" \
>>>>    op stop interval="0" timeout="120" \
>>>>    meta target-role="Started"
>>>>primitive ip1 ocf:heartbeat:IPaddr2 \
>>>>    params ip="10.10.10.193" nic="eth0" \
>>>>    op monitor interval="5s" \
>>>>    meta target-role="Started"
>>>>primitive ip1arp ocf:heartbeat:SendArp \
>>>>    params ip="10.10.10.193" nic="eth0" \
>>>>    meta target-role="Stopped"
>>>>primitive mysql ocf:heartbeat:mysql \
>>>>    params binary="/usr/bin/mysqld_safe" config="/etc/mysql/my.cnf"
>>>>user="mysql" group="mysql" log="/var/log/mysql.log"
>>>>pid="/var/run/mysqld/mysqld.pid" datadir="/var/lib/mysql"
>>>>socket="/var/run/mysqld/mysqld.sock" \
>>>>    op monitor interval="30s" timeout="30s" \
>>>>    op start interval="15" timeout="120" \
>>>>    op stop interval="0" timeout="120" \
>>>>    meta target-role="Started"
>>>>group MySQLDB fs_mysql ip1 mysql \
>>>>    meta target-role="Started"
>>>>ms ms_drbd_mysql drbd_mysql \
>>>>    meta master-max="1" master-node-max="1" clone-max="2"
>>>>clone-node-max="1" notify="true"
>>>>colocation fs_on_drbd inf: fs_mysql ms_drbd_mysql:Master colocation
>>>>mysql_on_drbd inf: MySQLDB fs_mysql order fs-mysql-after-drbd inf:
>>>>ms_drbd_mysql:promote fs_mysql:start order ip1-after-fs-mysql inf:
>>>>fs_mysql:start ip1:start order mysql-after-fs-mysql inf: fs_mysql:start
>>>>mysql:start property $id="cib-bootstrap-options" \
>>>>    dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \
>>>>    cluster-infrastructure="Heartbeat" \
>>>>    expected-quorum-votes="1" \
>>>>    stonith-enabled="false" \
>>>>    no-quorum-policy="ignore"
>>>>rsc_defaults $id="rsc-options" \
>>>>    resource-stickiness="100"
>>>>
>>>>Liang Ma
>>>>Contractuel | Consultant | SED Systems Inc.
>>>>Ground Systems Analyst
>>>>Agence spatiale canadienne | Canadian Space Agency
>>>>6767, Route de l'Aéroport, Longueuil (St-Hubert), QC, Canada, J3Y 8Y9
>>>>Tél/Tel : (450) 926-5099 | Téléc/Fax: (450) 926-5083
>>>>Courriel/E-mail : [liang.ma at space.gc.ca]
>>>>Site web/Web site : [www.space.gc.ca ]
>>>>
>>>>
>>>>
>>>>Hi,
>>>>
>>>>first of all, you configuration is, well, unconventional. I'd put all
>>>>the
>>>>primitives together in one group and any make the group colocated and
>>>>ordered in respect to the DRBD's. Perhaps it'd be wise to make two
>>>>groups.
>>>>
>>>>Googeling through the archives of the list I'd bet this error is caused
>>>>be the crm trying to mount a secondary DRBD. This might happen by some
>>>>constraints that somehow end up forming a loop.
>>>>
>>>>Could you please start with a very simple setup like:
>>>>
>>>>primitive resDRBD ocf:linbit:drbd params drbd_resource="r0"
>>>>primitive resFS ocf:heartbeat:Filesystem \
>>>>  params device="/dev/drbd0" directory="/mnt" fstype="ext4"
>>>>ms msDRBD resDRBD meta notify="true"
>>>>collocation col_FS_DRBD inf: resFS:Started msDRBD:Master order
>>>>ord_DRBD_FS inf: msDRBD:promote resFS:start
>>>>
>>>>If this works try to add a IP-Address as resource and make a group of
>>>>both
>>>>primitives:
>>>>
>>>>primitive resIP ocf:heartbeat:IPaddr2 \
>>>>  params ip="10.10.10.193" nic="eth0" cidr_netmask="24"
>>>>group groupMySQL resFS resIP
>>>>
>>>>Failover still working? What are the constraints now?
>>>>Now add the MySQL database to the group:
>>>>
>>>>primitive mysql ocf:heartbeat:mysql \
>>>>  params binary="/usr/bin/mysqld_safe" config="/etc/mysql/my.cnf" \
>>>>    user="mysql" group="mysql" log="/var/log/mysql.log" \
>>>>    pid="/var/run/mysqld/mysqld.pid" datadir="/var/lib/mysql" \
>>>>  socket="/var/run/mysqld/mysqld.sock"
>>>>
>>>>edit group groupMySQL
>>>>to Add the Mysql server
>>>>
>>>>and so on,
>>>>
>>>>
>>>>Hope you are successful taking one step after the other.
>>>>
>>>>Greetings,
>>>>--
>>>>Dr. Michael Schwartzkopff
>>>>Guardinistr. 63
>>>>81375 München
>>>>
>>>>Tel: (0163) 172 50 98
>>>>
>>>>-----Original Message-----
>>>>From: Ma, Liang
>>>>Sent: January 31, 2011 9:41 AM
>>>>To: The Pacemaker cluster resource manager
>>>>Subject: RE: [Pacemaker] pacemaker won't start mysql in the second node
>>>>
>>>>Hi,
>>>>
>>>>Thanks for your hints. I went through the cluster logs more carefully.
>>>>By
>>>>comparing the logs from the two nodes, the real different is after the
>>>>line 
>>>>
>>>>info: process_lrm_event: LRM operation fs_mysql_start_0
>>>>
>>>>On node arsvr1, after that line we got a confirmation on Action
>>>>fs_mysql_start_0 as such
>>>>
>>>>info: match_graph_event: Action fs_mysql_start_0 (8) confirmed on
>>>>arsvr1
>>>>
>>>>and then went on to Initiating action 9: start mysql_start_0 on arsvr1
>>>>(local).
>>>>
>>>>However on node arsvr2, we never see the confirmation from Action
>>>>fs_mysql_start_0. So mysql_start_0 is never called. But the strange
>>>>thing
>>>>is, I can see the drbd partition of fs_mysql is properly mounted on
>>>>arsvr2. Anyone knows what might stop arsvr2 to run that Action
>>>>fs_mysql_start_0 (8) confirmed?
>>>>
>>>>Thanks in advance.
>>>>
>>>>Here are the logs from the two nodes.
>>>>
>>>>Logs on Node 2:
>>>>
>>>>Jan 28 14:24:23 arsvr2 lrmd: [919]: info: rsc:fs_mysql:229: start Jan
>>>>28
>>>>14:24:23 arsvr2 Filesystem[1568]: [1596]: INFO: Running start for
>>>>/dev/drbd/by-res/r0 on /var/lib/mysql Jan 28 14:24:23 arsvr2 lrmd:
>>>>[919]:
>>>>info: RA output:
>>>>(fs_mysql:start:stderr) FATAL: Module scsi_hostadapter not found.
>>>>Jan 28 14:24:23 arsvr2 Filesystem[1568]: [1606]: INFO: Starting
>>>>filesystem check on /dev/drbd/by-res/r0 Jan 28 14:24:23 arsvr2 lrmd:
>>>>[919]: info: RA output:
>>>>(fs_mysql:start:stdout) fsck from util-linux-ng 2.17.2 Jan 28 14:24:23
>>>>arsvr2 lrmd: [919]: info: RA output:
>>>>(fs_mysql:start:stdout) /dev/drbd0: clean, 178/3276800 files,
>>>>257999/13106791 blocks
>>>>Jan 28 14:24:23 arsvr2 crmd: [922]: info: process_lrm_event: LRM
>>>>operation fs_mysql_start_0 (call=229, rc=0, cib-update=251,
>>>>confirmed=true) ok
>>>>Jan 28 14:24:46 arsvr2 cib: [918]: info: cib_stats: Processed 149
>>>>operations (0.00us average, 0% utilization) in the last 10min
>>>>
>>>>Logs on Node 1:
>>>>
>>>>Jan 28 14:28:58 arsvr1 lrmd: [1065]: info: rsc:fs_mysql:867: start Jan
>>>>28
>>>>14:28:58 arsvr1 crmd: [1068]: info: te_rsc_command: Initiating action
>>>>31:
>>>>monitor drbd_mysql:1_monitor_15000 on arsvr2 Jan 28 14:28:58 arsvr1
>>>>Filesystem[516]: [544]: INFO: Running start for /dev/drbd/by-res/r0 on
>>>>/var/lib/mysql Jan 28 14:28:58 arsvr1 lrmd: [1065]: info: RA output:
>>>>(fs_mysql:start:stderr) FATAL: Module scsi_hostadapter not found.
>>>>Jan 28 14:28:58 arsvr1 Filesystem[516]: [554]: INFO: Starting
>>>>filesystem
>>>>check on /dev/drbd/by-res/r0 Jan 28 14:28:58 arsvr1 lrmd: [1065]: info:
>>>>RA output:
>>>>(fs_mysql:start:stdout) fsck from util-linux-ng 2.17.2 Jan 28 14:28:58
>>>>arsvr1 lrmd: [1065]: info: RA output:
>>>>(fs_mysql:start:stdout) /dev/drbd0: clean, 178/3276800 files,
>>>>257999/13106791 blocks
>>>>Jan 28 14:28:58 arsvr1 crmd: [1068]: info: process_lrm_event: LRM
>>>>operation fs_mysql_start_0 (call=867, rc=0, cib-update=1650,
>>>>confirmed=true) ok
>>>>Jan 28 14:28:58 arsvr1 crmd: [1068]: info: match_graph_event: Action
>>>>fs_mysql_start_0 (8) confirmed on arsvr1 (rc=0)
>>>>Jan 28 14:28:58 arsvr1 crmd: [1068]: info: te_rsc_command: Initiating
>>>>action 9: start mysql_start_0 on arsvr1 (local)
>>>>Jan 28 14:28:58 arsvr1 crmd: [1068]: info: do_lrm_rsc_op: Performing
>>>>key=9:551:0:9c402121-906c-42de-a18a-68deb24208cb op=mysql_start_0 )
>>>>Jan 28 14:28:58 arsvr1 lrmd: [1065]: info: rsc:mysql:868: start
>>>>Jan 28 14:28:58 arsvr1 mysqld_safe: Starting mysqld daemon with
>>>>databases
>>>>from /var/lib/mysql
>>>>Jan 28 14:28:59 arsvr1 crmd: [1068]: info: match_graph_event: Action
>>>>drbd_mysql:1_monitor_15000 (31) confirmed on arsvr2 (rc=0)
>>>>Jan 28 14:29:02 arsvr1 mysql[576]: [728]: INFO: MySQL started
>>>>Jan 28 14:29:02 arsvr1 crmd: [1068]: info: process_lrm_event: LRM
>>>>operation mysql_start_0 (call=868, rc=0, cib-update=1651,
>>>>confirmed=true) ok
>>>>Jan 28 14:29:02 arsvr1 crmd: [1068]: info: match_graph_event: Action
>>>>mysql_start_0 (9) confirmed on arsvr1 (rc=0)
>>>>
>>>>
>>>>Liang Ma
>>>>Contractuel | Consultant | SED Systems Inc.
>>>>Ground Systems Analyst
>>>>Agence spatiale canadienne | Canadian Space Agency
>>>>6767, Route de l'Aéroport, Longueuil (St-Hubert), QC, Canada, J3Y 8Y9
>>>>Tél/Tel : (450) 926-5099 | Téléc/Fax: (450) 926-5083
>>>>Courriel/E-mail : [liang.ma at space.gc.ca]
>>>>Site web/Web site : [www.space.gc.ca ]
>>>>
>>>>
>>>>
>>>>
>>>>-----Original Message-----
>>>>From: Dejan Muhamedagic [mailto:dejanmm at fastmail.fm]
>>>>Sent: January 28, 2011 11:09 AM
>>>>To: The Pacemaker cluster resource manager
>>>>Subject: Re: [Pacemaker] pacemaker won't start mysql in the second node
>>>>
>>>>On Fri, Jan 28, 2011 at 08:50:45AM -0500, Liang.Ma at asc-csa.gc.ca wrote:
>>>>> Hi Dejan, thanks for your reply.
>>>>> 
>>>>> That's one of the problem. I don't see any logs in log file
>>>>>/var/log/mysql/error.log.
>>>>
>>>>I meant the cluster logs.
>>>>
>>>>> I checked the permission of directories /var/run/mysqld and
>>>>>/var/log/mysql. In both nodes they are the same as
>>>>> 
>>>>> drwxr-xr-x 2 mysql root 40 2011-01-27 13:50 /var/run/mysqld/
>>>>> drwxr-s--- 2 mysql adm 4096 2011-01-27 11:34 /var/log/mysql
>>>>> 
>>>>> By the way, under which user pacemaker runs, root or someone else?
>>>>
>>>>pacemaker is a collection of programs. At any rate, the RA run
>>>>as root, but may su to another user (mysql) depending on the
>>>>resource configuration.
>>>>
>>>>Thanks,
>>>>
>>>>Dejan
>>>>
>>>>> Liang Ma
>>>>> Contractuel | Consultant | SED Systems Inc.
>>>>> Ground Systems Analyst
>>>>> Agence spatiale canadienne | Canadian Space Agency
>>>>> 6767, Route de l'Aéroport, Longueuil (St-Hubert), QC, Canada, J3Y 8Y9
>>>>> Tél/Tel : (450) 926-5099 | Téléc/Fax: (450) 926-5083
>>>>> Courriel/E-mail : [liang.ma at space.gc.ca]
>>>>> Site web/Web site : [www.space.gc.ca ]
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> -----Original Message-----
>>>>> From: Dejan Muhamedagic [mailto:dejanmm at fastmail.fm]
>>>>> Sent: January 28, 2011 8:26 AM
>>>>> To: The Pacemaker cluster resource manager
>>>>> Subject: Re: [Pacemaker] pacemaker won't start mysql in the second
>>>>>node
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> On Thu, Jan 27, 2011 at 11:51:31AM -0500, Liang.Ma at asc-csa.gc.ca
>>>>>wrote:
>>>>> > 
>>>>> > 
>>>>> > Hi There,
>>>>> > 
>>>>> > I have set up a pair of ha LAMP servers using heartbeat, pacemaker
>>>>>and
>>>>> > drbd on Ubuntu 10.04 LTS. Everything works fine until I upgraded
>>>>> > mysql-server from 5.1.41-3ubuntu12.6 to 5.1.41-3ubuntu12.9. Now
>>>>>node
>>>>>1
>>>>> > (arsvr1) works still fine, but mysql on node 2 (arsvr2) won't start
>>>>> > when I switch arsvr1 standby. The error message shown from "crm
>>>>> > status" is
>>>>> > 
>>>>> > Failed actions:
>>>>> > mysql_start_0 (node=arsvr2, call=32, rc=4, status=complete):
>>>>> > insufficient privileges
>>>>> > 
>>>>> > No errors logged in /var/log/mysql/error.log at all.
>>>>> 
>>>>> I think that you should check directory permissions. The log
>>>>> file should give you a hint.
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Dejan
>>>>> 
>>>>> 
>>>>> > drbd mysql partition mounted properly. If I go to
>>>>> > /usr/lib/ocf/resource.d/heartbeat and set the OCF_RESKEY
>>>>>parameters,
>>>>>I
>>>>> > have no problem to start mysql server by "./mysql start". But the
>>>>> > resource mysql won't show up in crm status.
>>>>> > 
>>>>> > So looks somehow pacemaker fail to start resource mysql even before
>>>>> > running the resource script.
>>>>> > 
>>>>> > Here is the configuration
>>>>> > 
>>>>> > node $id="bc6bf61d-6b5f-4307-85f3-bf7bb11531bb" arsvr2 \
>>>>> >     attributes standby="off"
>>>>> > node $id="bf0e7394-9684-42b9-893b-5a9a6ecddd7e" arsvr1 \
>>>>> >     attributes standby="off"
>>>>> > primitive apache2 lsb:apache2 \
>>>>> >     op start interval="0" timeout="60" \
>>>>> >     op stop interval="0" timeout="120" start-delay="15" \
>>>>> >     meta target-role="Started"
>>>>> > primitive drbd_mysql ocf:linbit:drbd \
>>>>> >     params drbd_resource="r0" \
>>>>> >     op monitor interval="15s"
>>>>> > primitive drbd_webfs ocf:linbit:drbd \
>>>>> >     params drbd_resource="r1" \
>>>>> >     op monitor interval="15s" \
>>>>> >     op start interval="0" timeout="240" \
>>>>> >     op stop interval="0" timeout="100"
>>>>> > primitive fs_mysql ocf:heartbeat:Filesystem \
>>>>> >     params device="/dev/drbd/by-res/r0" directory="/var/lib/mysql"
>>>>>fstype="ext4" \
>>>>> >     op start interval="0" timeout="60" \
>>>>> >     op stop interval="0" timeout="120" \
>>>>> >     meta target-role="Started"
>>>>> > primitive fs_webfs ocf:heartbeat:Filesystem \
>>>>> >     params device="/dev/drbd/by-res/r1" directory="/srv"
>>>>>fstype="ext4" \
>>>>> >     op start interval="0" timeout="60" \
>>>>> >     op stop interval="0" timeout="120" \
>>>>> >     meta target-role="Started"
>>>>> > primitive ip1 ocf:heartbeat:IPaddr2 \
>>>>> >     params ip="10.10.10.193" nic="eth0" \
>>>>> >     op monitor interval="5s"
>>>>> > primitive ip1arp ocf:heartbeat:SendArp \
>>>>> >     params ip="10.10.10.193" nic="eth0"
>>>>> > primitive mysql ocf:heartbeat:mysql \
>>>>> >     params binary="/usr/bin/mysqld_safe" config="/etc/mysql/my.cnf"
>>>>> > user="mysql" group="mysql" log="/var/log/mysql.log"
>>>>> > pid="/var/run/mysqld/mysqld.pid" datadir="/var/lib/mysql"
>>>>> > socket="/var/run/mysqld/mysqld.sock" \
>>>>> >     op monitor interval="30s" timeout="30s" \
>>>>> >     op start interval="0" timeout="120" \
>>>>> >     op stop interval="0" timeout="120" \
>>>>> >     meta target-role="Started"
>>>>> > group MySQLDB fs_mysql mysql \
>>>>> >     meta target-role="Started"
>>>>> > group WebServices ip1 ip1arp fs_webfs apache2 \
>>>>> >     meta target-role="Started"
>>>>> > ms ms_drbd_mysql drbd_mysql \
>>>>> >     meta master-max="1" master-node-max="1" clone-max="2"
>>>>> > clone-node-max="1" notify="true"
>>>>> > ms ms_drbd_webfs drbd_webfs \
>>>>> >     meta master-max="1" master-node-max="1" clone-max="2"
>>>>> > clone-node-max="1" notify="true" target-role="Started"
>>>>> > colocation apache2_with_ip inf: apache2 ip1
>>>>> > colocation apache2_with_mysql inf: apache2 ms_drbd_mysql:Master
>>>>> > colocation apache2_with_webfs inf: apache2 ms_drbd_webfs:Master
>>>>> > colocation fs_on_drbd inf: fs_mysql ms_drbd_mysql:Master
>>>>> > colocation ip_with_ip_arp inf: ip1 ip1arp
>>>>> > colocation mysql_on_drbd inf: MySQLDB ms_drbd_mysql:Master
>>>>> > colocation web_with_mysql inf: MySQLDB WebServices
>>>>> > colocation webfs_on_drbd inf: fs_webfs ms_drbd_webfs:Master
>>>>> > colocation webfs_with_fs inf: fs_webfs fs_mysql
>>>>> > order apache2-after-arp inf: ip1arp:start apache2:start
>>>>> > order arp-after-ip inf: ip1:start ip1arp:start
>>>>> > order fs-mysql-after-drbd inf: ms_drbd_mysql:promote fs_mysql:start
>>>>> > order fs-webfs-after-drbd inf: ms_drbd_webfs:promote fs_webfs:start
>>>>> > order ip-after-mysql inf: mysql:start ip1:start
>>>>> > order mysql-after-fs-mysql inf: fs_mysql:start mysql:start
>>>>> > property $id="cib-bootstrap-options" \
>>>>> >     dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \
>>>>> >     cluster-infrastructure="Heartbeat" \
>>>>> >     expected-quorum-votes="1" \
>>>>> >     stonith-enabled="false" \
>>>>> >     no-quorum-policy="ignore"
>>>>> > rsc_defaults $id="rsc-options" \
>>>>> >     resource-stickiness="100"
>>>>> > 
>>>>> > Any help please?
>>>>> > 
>>>>> > Thanks,
>>>>> > 
>>>>> > Liang Ma
>>>>> > Contractuel | Consultant | SED Systems Inc.
>>>>> > Ground Systems Analyst
>>>>> > Agence spatiale canadienne | Canadian Space Agency
>>>>> > 6767, Route de l'Aéroport, Longueuil (St-Hubert), QC, Canada, J3Y
>>>>>8Y9
>>>>> > Tél/Tel : (450) 926-5099 | Téléc/Fax: (450) 926-5083
>>>>> > Courriel/E-mail : [liang.ma at space.gc.ca]
>>>>> > Site web/Web site : [www.space.gc.ca ]
>>>>> > 
>>>>> > 
>>>>> > 
>>>>> > 
>>>>> > _______________________________________________
>>>>> > Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>> > http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>> > 
>>>>> > Project Home: http://www.clusterlabs.org
>>>>> > Getting started:
>>>>>http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>> > Bugs: 
>>>>>http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacema
>>>>>k
>>>>>e
>>>>>r
>>>>> 
>>>>> _______________________________________________
>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>> 
>>>>> Project Home: http://www.clusterlabs.org
>>>>> Getting started:
>>>>>http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>> Bugs: 
>>>>>http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacema
>>>>>k
>>>>>e
>>>>>r
>>>>> 
>>>>> _______________________________________________
>>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>> 
>>>>> Project Home: http://www.clusterlabs.org
>>>>> Getting started:
>>>>>http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>> Bugs: 
>>>>>http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacema
>>>>>k
>>>>>e
>>>>>r
>>>>
>>>>_______________________________________________
>>>>Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>>Project Home: http://www.clusterlabs.org
>>>>Getting started:
>>>>http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>Bugs: 
>>>>http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemak
>>>>e
>>>>r
>>>>
>>>>_______________________________________________
>>>>Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>>http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>>Project Home: http://www.clusterlabs.org
>>>>Getting started:
>>>>http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>Bugs: 
>>>>http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemak
>>>>e
>>>>r
>>>
>>>
>>>
>>>_______________________________________________
>>>Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>>Project Home: http://www.clusterlabs.org
>>>Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>Bugs: 
>>>http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemake
>>>r
>>>
>>>_______________________________________________
>>>Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>>Project Home: http://www.clusterlabs.org
>>>Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>Bugs: 
>>>http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemake
>>>r
>>
>>
>>
>>_______________________________________________
>>Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>>Project Home: http://www.clusterlabs.org
>>Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>Bugs: 
>>http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>>
>>_______________________________________________
>>Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>>Project Home: http://www.clusterlabs.org
>>Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>Bugs: 
>>http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>
>
>
>_______________________________________________
>Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
>Project Home: http://www.clusterlabs.org
>Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>Bugs: 
>http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>
>_______________________________________________
>Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
>Project Home: http://www.clusterlabs.org
>Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>Bugs: 
>http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker

More information about the Pacemaker mailing list