[Pacemaker] drbd 8.3.2 stacked resources controlled by pacemaker
Dan Frincu
dfrincu at streamwide.ro
Tue Sep 22 22:30:09 UTC 2009
bump
anyone?
Dan Frincu wrote:
> Hello all,
>
> I'm trying to build an architecture that I'm not 100% sure will work
> so I need some input on the matter. The design is:
> - 4 servers (now running on test xen virtual machines)
> - the servers will be divided in two separate geographical locations,
> 2 servers in site A and 2 servers in site B
> - each pair of servers from the same site will form a cluster
> - each cluster from both sites will be linked as a stacked resource
> (stacked-on-top-of)
>
> For this setup I'm using drbd-8.3.2-3.x86_64.rpm,
> heartbeat-3.0.0-33.2.x86_64.rpm, openais-0.80.5-15.1.x86_64.rpm,
> resource-agents-1.0-31.4.x86_64.rpm and their respective dependencies.
>
> One question that popped to my mind was that pacemaker is using
> multicast, so the connection between the two sites (if it is done
> through the public internet) should involve multicast routing? What
> bandwidth requirements should be for this kind of setup? I'm assuming
> that the high end requirements will be on the drbd replication part,
> but also I'm interested if there are any specific requirements related
> to latency, delay and (probably) jitter on the multicast connection as
> it uses (correct me if I'm wrong) UDP as a transport?
>
> So far I've configured all 4 virtual machines with drbd, created
> "normal" resources, stacked resources and all seems to be in order,
> the problem is that I don't know how does pacemaker deal with stacked
> resources.
>
> I mean, the goal here is to have one service, let's say apache, run on
> a device on the stacked resource and to be handled by pacemaker so
> that the apache server either runs on primary stacked resource on site
> A, or fails over on stacked resource on site B, and underneath all
> that, if in site A one of the two servers fails, it switches to the
> other one and the same should happen in site B.
>
> I see it like a RAID-1 "array" on top of two RAID-1 "arrays", if you
> understand the analogy.
>
> cat /etc/ais/openais.conf
> aisexec {
> user: root
> group: root
> }
>
> amf {
> mode: disabled
> }
>
> logging {
> to_stderr: yes
> debug: off
> timestamp: on
> to_file: no
> to_syslog: yes
> syslog_facility: daemon
> }
>
> totem {
> version: 2
> token: 3000
> token_retransmits_before_loss_const: 10
> join: 60
> consensus: 1500
> vsftype: none
> max_messages: 20
> clear_node_high_bit: yes
> secauth: off
> threads: 0
> # nodeid: 1234
> rrp_mode: passive
>
> interface {
> ringnumber: 0
> bindnetaddr: 192.168.1.0
> mcastaddr: 226.94.1.1
> mcastport: 5405
> }
>
> interface {
> ringnumber: 1
> bindnetaddr: 172.16.0.0
> mcastaddr: 226.94.1.1
> mcastport: 5406
> }
> }
>
> service {
> ver: 0
> name: pacemaker
> use_mgmtd: no
> }
>
> A question on the openais.conf, these two "rings", do they have to be
> on separate subnets or can they be on the same subnet. On the same
> issue, should one interface declaration be linked to the subnet where
> drbd is also running and the other declaration to be linked to
> pacemaker, or it doesn't matter which goes where?
>
> In /etc/drbd.conf the stacked resources look like this:
>
> resource "st1" {
> protocol C;
>
> on srv1 {
> device /dev/drbd0;
> disk /dev/xvda3;
> address 172.16.0.1:7700;
> flexible-meta-disk internal;
> }
>
> on srv2 {
> device /dev/drbd0;
> disk /dev/xvda3;
> address 172.16.0.2:7700;
> flexible-meta-disk internal;
> }
> }
>
> resource "st2" {
> protocol C;
>
> on srv3 {
> device /dev/drbd0;
> disk /dev/xvda3;
> address 172.16.0.3:7701;
> flexible-meta-disk internal;
> }
>
> on srv4 {
> device /dev/drbd0;
> disk /dev/xvda3;
> address 172.16.0.4:7701;
> flexible-meta-disk internal;
> }
> }
>
> resource "stacked" {
> protocol C;
> stacked-on-top-of st1 {
> device /dev/drbd10;
> address 172.16.0.1:7704;
> }
>
> stacked-on-top-of st2 {
> device /dev/drbd10;
> address 172.16.0.3:7704;
> }
> }
>
> The cat /proc/drbd shows the resources both normal and stacked are up
> and running:
>
> 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----
> ns:1 nr:0 dw:1 dr:110 al:1 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
>
> 10: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
>
> Now in order for me to start the resources, I have to perform the
> start process manually, involving /etc/init.d/drbd start, which I have
> read in the docs that isn't recommended if I want to use OCF style
> resources, which I do.
> I first start drbd, this loads the normal resources, which stay in a
> "cs:Connected ro:Secondary/Secondary" state until I do a "drbdadm
> --primary all". Then I have to do a "drbdadm --stacked up stacked" on
> the cluster members I've assigned as primary, then the stacked
> resources go into a "cs:Connected ro:Secondary/Secondary" state until
> I, again, set one of them as primary.
> So the question is, how do I get pacemaker to start both normal and
> stacked resources, promote normal and stacked resources as primary and
> then start a web server on the stacked resource that becomes primary.
>
> Any relevant documentation links or examples are more than welcome, as
> well as ideas and suggestions.
>
> Thank you in advance,
> Dan.
>
--
Dan FRINCU
Streamwide Romania
E-mail: dfrincu at streamwide.ro
More information about the Pacemaker
mailing list