[ClusterLabs] SAN with drbd and pacemaker

Mon Sep 21 07:14:24 UTC 2015

"With 20 disk of 4TB you have a total capacity of 80TB. If you run all of
them as RAID6 then you have a total of 72TB."

And that's the point! I'm trying to understand if I can create more RAID6
arrays and how my controller handles disk failures in that case. First I
think we need to clarify terminology related to Megaraid Storage Manager
and for this reason I attach here a screenshot -> (phyical drives ->
http://pasteboard.co/NC3O60x.png    and logical drives ->
http://pasteboard.co/NC8DLcM.png )
So, reading this ->
http://www.cisco.com/c/dam/en/us/td/docs/unified_computing/ucs/3rd-party/lsi/mrsas/userguide/LSI_MR_SAS_SW_UG.pdf
(page 41) I think I have a RAID (6 in my case) array as a drive group and a
volume as a Virtual Drive. If this is right, I should discover how much
RAID array my controller supports. Actually I have 20 disks, but I can add
more. However, reducing rebuild array time is my goal, so I think that
create a virtual drive for each drive group is the right way. Please give
me some advises....
Thanks

2015-09-18 13:02 GMT+02:00 Kai Dupke <kdupke at suse.com>:

> On 09/18/2015 09:28 AM, Marco Marino wrote:
> > Can you explain me this? 16 volumes?
>
>
> With 20 disk of 4TB you have a total capacity of 80TB. If you run all of
> them as RAID6 then you have a total of 72TB.
>
> If you ask your controller to create a 8TB volume, this volume is spread
> across all the 20 disk. As 2 stripes are used for parity, you have
> 20-2=18 data stripes per volume. This makes each stripe 444G big,
> leaving 3500G free for other volumes.
>
> If you fill up the remaining 3500G with volumes the same way, you get 8
> additional volumes (OK, the last volume is <8TB then).
>
> In total you have 9 volumes then, each disk has data/parity on all of
> these volumes.
>
> 9x8=72, voila!
>
> If a disk error appear and the controller marks the disk dead then all 9
> volumes are affected.
>
> With 20 6TB/8TB drives, you just get more 8TB volumes using this way.
>
> What would of course reduce the risk is to always use <20 disk in one
> raid6 volume, so not each disk serves all volumes.
>
> Another issue is about performance, not every RAID controller performs
> best with 20 drives. Adaptec recommends an odd number of drives, with 7
> or 9 drives performs best AFAIK.
>
> So you could make volume 1 on disks 1-9, volume 2 on disk 2-10, volume 3
> on disk 3-11 etc. etc.
>
> Or consider using some combination of RAID6 and RAID1, but this gives
> you way less available disk size (and no, I have no calculation handy on
> the chance for failure for RAID6 vs. RAID15 vs. RAID16)
>
> greetings kai
>
>
> >
> > Thank you
> >
> >
> >
> > 2015-09-17 15:54 GMT+02:00 Kai Dupke <kdupke at suse.com>:
> >
> >> On 09/17/2015 09:44 AM, Marco Marino wrote:
> >>> Hi, I have 2 servers supermicro lsi 2108 with many disks (80TB) and I'm
> >>> trying to build a SAN with drbd and pacemaker. I'm studying, but I have
> >> no
> >>> experience on large array of disks with drbd and pacemaker, so I have
> >> some
> >>> questions:
> >>>
> >>> I'm using MegaRAID Storage Manager to create virtual drives. Each
> virtual
> >>> drive is a device on linux (eg /dev/sdb, /dev/sdc.....), so my first
> >>> question is: it's a good idea to create virtual drive of 8 TB (max)?
> I'm
> >>> thinking to rebuild array time in case of disk failure (about 1 day
> for 8
> >>
> >> It depends on your disks and RAID level. If one disk fails the content
> >> of this disk has to be recreated by either copying (all RAID levels with
> >> some RAID 1 included) or calculating (all with no RAID1 included), in
> >> the later case all disks get really stressed.
> >>
> >> If you run 20x4TB disks as RAID6, then an 8TB volume is only ~500G per
> >> disk. However, if one disk fails, then all the other 15 volumes this
> >> disk handles are broken, too. (BTW, most raid controller can handle
> >> multiple stripes per disk, but usually only a handful) In such case the
> >> complete 4TB of the broken disk has to be recovered, affecting all 16
> >> volumes.
> >>
> >> On the other side, if you use 4x5x4TB as 4x 12TB RAID6, a broken disk
> >> only affects one of 4 volumes - but at the cost of more disks needed.
> >>
> >> You can do the similar calculation based on RAID16/15.
> >>
> >> The only reason I see to create small slices is to make them fit on
> >> smaller replacement disks, which might be more easily available/payable
> >> at time of error (but now we are entering a more low cost area where
> >> usually SAN and DRBD do not take place).
> >>
> >> greetings
> >> Kai Dupke
> >> Senior Product Manager
> >> Server Product Line
> >> --
> >> Sell not virtue to purchase wealth, nor liberty to purchase power.
> >> Phone:  +49-(0)5102-9310828     Mail: kdupke at suse.com
> >> Mobile: +49-(0)173-5876766      WWW:  www.suse.com
> >>
> >> SUSE Linux GmbH - Maxfeldstr. 5 - 90409 Nuernberg (Germany)
> >> GF:Felix Imendörffer,Jane Smithard,Graham Norton,HRB 21284 (AG Nürnberg)
> >>
> >> _______________________________________________
> >> Users mailing list: Users at clusterlabs.org
> >> http://clusterlabs.org/mailman/listinfo/users
> >>
> >> Project Home: http://www.clusterlabs.org
> >> Getting started:
> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> >> Bugs: http://bugs.clusterlabs.org
> >>
> >
> >
> >
> > _______________________________________________
> > Users mailing list: Users at clusterlabs.org
> > http://clusterlabs.org/mailman/listinfo/users
> >
> > Project Home: http://www.clusterlabs.org
> > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> > Bugs: http://bugs.clusterlabs.org
> >
>
>
>
> Kai Dupke
> Senior Product Manager
> Server Product Line
> --
> Sell not virtue to purchase wealth, nor liberty to purchase power.
> Phone:  +49-(0)5102-9310828     Mail: kdupke at suse.com
> Mobile: +49-(0)173-5876766      WWW:  www.suse.com
>
> SUSE Linux GmbH - Maxfeldstr. 5 - 90409 Nuernberg (Germany)
> GF:Felix Imendörffer,Jane Smithard,Graham Norton,HRB 21284 (AG Nürnberg)
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20150921/0b0c06a1/attachment.htm>