[Pacemaker] Trouble with DRBD mount
senrabdet at aol.com
senrabdet at aol.com
Thu Feb 28 07:19:20 EST 2013
Hi All:
We are stuck trying to get pacemaker to work with DRBD, and having tried various alternatives can't get our "drbd1" to mount and get some errors.
NOTE: we are trying to get pacemaker to work with an existing Encrypted RAID1 LVM setup - is this impossible or a "just plain bad idea"? We were thinking we'd like the potential advantages of local RAID on each box as well as the Internet RAID & failover provided by DRBD/pacemaker. We're using Debian Squeeze. Per various instructions, we've disabled the DRBD boot init (update-rc.d -f drbd remove) and set the LVM filter to filter = [ "a|drbd.*|", "r|.*|" ].
FYI - we've commented out the LVM mount "/dev/vg2/vserverLV" in our fstab, and consistently seem to need to do this to avoid a boot error.
We think DRBD works until we add in the pacemaker steps (i.e., "dev/drbd1" mounts at boot; we can move related data from server1 to server2 back and forth, though need to use the command line to accomplish this). We've seen various statements on the net that suggest it is viable to use a "mapper" disk choice in drbd.conf. Also, if we start by configuring Pacemaker for a simple IP failover, that works (i.e., no errors, we can ping via the fail over address) but stops working when we add in the DRBD primatives and related statements. Our suspicion (other than maybe "you can't do this with existing RAID") is that we're using the wrong "disk" statement in our drbd.conf or maybe in our "primitive fs_vservers" statement, though we've tried lots of alternatives and this is the same drbd.conf we use before adding in Pacemaker and it seems to work at that point.
Lastly, while various config statements refer to "vservers", we have not gotten to the point of trying to add any data to the DRBD devices other than a few text files that have disappeared since doing our "crm" work.
Any help appreciated! Thanks, Ted
CONFIGS/LOGS
A) drbd.conf
global { usage-count no; }
common { syncer { rate 100M; } }
#original
resource r1 {
protocol C;
startup {
wfc-timeout 15;
degr-wfc-timeout 60;
}
device /dev/drbd1 minor 1;
disk /dev/vg2/vserverLV;
meta-disk internal;
# following 2 definition are equivalent
on server1 {
address 192.168.1.129:7801;
disk /dev/vg2/vserverLV;
}
on server2 {
address 192.168.1.128:7801;
disk /dev/vg2/vserverLV;
# disk /dev/mapper/md2_crypt;
}
# floating 192.168.5.41:7801;
# floating 192.168.5.42:7801;
net {
cram-hmac-alg sha1;
shared-secret "secret";
after-sb-0pri discard-younger-primary; #discard-zero-changes;
after-sb-1pri discard-secondary;
after-sb-2pri call-pri-lost-after-sb;
}
}
B) Pacemaker Config
crm configure show
node server1
node server2
primitive app_ip ocf:heartbeat:IPaddr \
params ip="192.168.1.152" \
op monitor interval="30s"
primitive drbd ocf:linbit:drbd \
params drbd_resource="r1" \
op start interval="0" timeout="240" \
op stop interval="0" timeout="100" \
op monitor interval="59s" role="Master" timeout="30s" \
op monitor interval="60s" role="Slave" timeout="30s"
primitive fs_vservers ocf:heartbeat:Filesystem \
params device="/dev/drbd1" directory="/vservers" fstype="ext4" \
op start interval="0" timeout="60" \
op stop interval="0" timeout="120"
primitive vg2 ocf:heartbeat:LVM \
params volgrpname="vg2" exclusive="true" \
op start interval="0" timeout="30" \
op stop interval="0" timeout="30"
group lvm app_ip vg2 fs_vservers
ms ms_drbd drbd \
meta master-node-max="1" clone-max="2" clone-node-max="1" globally-unique="false" notify="true" target-role="Master"
location drbd_on_node1 ms_drbd \
rule $id="drbd_on_node1-rule" $role="master" 100: #uname eq server1
colocation vserver-deps inf: ms_drbd:Master lvm
order app_on_drbd inf: ms_drbd:promote lvm:start
property $id="cib-bootstrap-options" \
dc-version="1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
stonith-enabled="false" \
no-quorum-policy="ignore"
C) crm status results (with errors)
Last updated: Wed Feb 27 19:05:57 2013
Stack: openais
Current DC: server1 - partition with quorum
Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b
2 Nodes configured, 2 expected votes
2 Resources configured.
============
Online: [ server2 server1 ]
Migration summary:
* Node server2:
drbd:1: migration-threshold=1000000 fail-count=1000000
* Node server1:
drbd:0: migration-threshold=1000000 fail-count=1000000
Failed actions:
drbd:1_start_0 (node=server2, call=8, rc=-2, status=Timed Out): unknown exec error
drbd:0_start_0 (node=server1, call=6, rc=-2, status=Timed Out): unknown exec error
D) Mount
/dev/mapper/vg1-root on / type ext4 (rw,errors=remount-ro)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
/dev/md0 on /boot type ext4 (rw)
/dev/mapper/vg1-home on /home type ext4 (rw)
/dev/mapper/vg1-tmp on /tmp type ext4 (rw)
/dev/mapper/vg1-usr on /usr type ext4 (rw)
/dev/mapper/vg1-var on /var type ext4 (rw)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
E) fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
/dev/mapper/vg1-root / ext4 errors=remount-ro 0 1
# /boot was on /dev/md0 during installation
UUID=25829c6c-164c-4a1e-9e84-6bab180e38f4 /boot ext4 defaults 0 2
/dev/mapper/vg1-home /home ext4 defaults 0 2
/dev/mapper/vg1-tmp /tmp ext4 defaults 0 2
/dev/mapper/vg1-usr /usr ext4 defaults 0 2
/dev/mapper/vg1-var /var ext4 defaults 0 2
#/dev/mapper/vg2-vserverLV /vservers ext4 defaults 0 2
/dev/mapper/vg1-swap none swap sw 0 0
/dev/scd0 /media/cdrom0 udf,iso9660 user,noauto 0 0
/dev/scd1 /media/cdrom1 udf,iso9660 user,noauto 0 0
/dev/fd0 /media/floppy0 auto rw,user,noauto 0 0
F) fdisk -l
Disk /dev/sda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0007c7a2
Device Boot Start End Blocks Id System
/dev/sda1 * 1 61 487424 fd Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sda2 61 1885 14648320 fd Linux raid autodetect
/dev/sda3 1885 3101 9765888 fd Linux raid autodetect
Disk /dev/sdb: 203.9 GB, 203928109056 bytes
255 heads, 63 sectors/track, 24792 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0008843c
Device Boot Start End Blocks Id System
/dev/sdb1 * 1 61 487424 fd Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sdb2 61 1885 14648320 fd Linux raid autodetect
/dev/sdb3 1885 3101 9765888 fd Linux raid autodetect
Disk /dev/md0: 499 MB, 499109888 bytes
2 heads, 4 sectors/track, 121853 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/md1: 15.0 GB, 14998757376 bytes
2 heads, 4 sectors/track, 3661806 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x08040000
Disk /dev/md2: 9999 MB, 9999147008 bytes
2 heads, 4 sectors/track, 2441198 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x08040000
Disk /dev/dm-0: 15.0 GB, 14997704704 bytes
255 heads, 63 sectors/track, 1823 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/dm-1: 3997 MB, 3997171712 bytes
255 heads, 63 sectors/track, 485 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/dm-2: 1996 MB, 1996488704 bytes
255 heads, 63 sectors/track, 242 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/dm-3: 1996 MB, 1996488704 bytes
255 heads, 63 sectors/track, 242 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/dm-4: 3997 MB, 3997171712 bytes
255 heads, 63 sectors/track, 485 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/dm-5: 1996 MB, 1996488704 bytes
255 heads, 63 sectors/track, 242 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/dm-6: 499 MB, 499122176 bytes
255 heads, 63 sectors/track, 60 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/dm-7: 9998 MB, 9998094336 bytes
255 heads, 63 sectors/track, 1215 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
G) syslog excerpt
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) Command '
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) drbdsetup
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr)
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) 1
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr)
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) disk
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr)
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) /dev/vg2/vserverLV
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr)
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) /dev/vg2/vserverLV
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr)
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) internal
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr)
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) --set-defaults
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr)
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) --create-device
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) ' terminated with exit code 20
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) drbdadm attach r1: exited with code 20
Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Called drbdadm -c /etc/drbd.conf --peer server2 up r1
Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Exit code 1
Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Command output:
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stdout)
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) Can not open device '/dev/vg2/vserverLV': No such file or directory
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) Command 'drbdsetup 1 disk /dev/vg2/vserverLV
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) /dev/vg2/vserverLV internal --set-defaults --create-device' terminated with exit code 20#012drbdadm attach r1: exited with code 20
Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Called drbdadm -c /etc/drbd.conf --peer server2 up r1
Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Exit code 1
Feb 27 06:36:22 server1 drbd[2329]: ERROR: r1: Command output:
Feb 27 06:36:22 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stdout)
Feb 27 06:36:23 server1 lrmd: [1705]: info: RA output: (p_drbd_r1:0:start:stderr) Can not open device '/dev/vg2/vserverLV': No such file or directory
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.clusterlabs.org/pipermail/pacemaker/attachments/20130228/a75a9b38/attachment-0002.html>
More information about the Pacemaker
mailing list