[ClusterLabs] Major problem with iSCSITarget resource on top of DRBD M/S resource.

[Digimer]

On 27/09/15 11:22 AM, Alex Crow wrote:
> 
> 
> On 27/09/15 16:06, Digimer wrote:
>> On 27/09/15 11:02 AM, Alex Crow wrote:
>>>
>>> On 27/09/15 15:54, Digimer wrote:
>>>> On 27/09/15 10:40 AM, Alex Crow wrote:
>>>>> Hi List,
>>>>>
>>>>> I'm trying to set up a failover iSCSI storage system for oVirt using a
>>>>> self-hosted engine. I've set up DRBD in Master-Slave for two iSCSI
>>>>> targets, one for the self-hosted engine and one for the VMs. I has
>>>>> this
>>>>> all working perfectly, then after trying to move the engine's LUN
>>>>> to the
>>>>> opposite host, all hell broke loose. The VMS LUN is still fine, starts
>>>> I'm guessing no fencing?
>>> Hi Digimer,
>>>
>>> No, but I've tried turning off one machine and still no success as a
>>> single node :-(
>> You *must* have working fencing, anyway. So now strikes me as a
>> fantastic time to add it. Turning off the node alone doesn't help.
>>
> 
> I did a pcs stonith confirm after sending magic-sysrq-b to glenrock and
> it looks like it's up now, cheers very much!
> 
> I do have some APC power bars, but they are APC-rebranded Schneider kit,
> which is frankly awful, not at all as good as the old APC stuff. BTW
> avoid APC KVMs like the plague, they seriously suck - hanging all the
> time, refuse to allow a console view from any other than the same
> network range they are on, the list is huge. Support is also dire.
> 
> I might have to use the fallback of IPMI but it don't like it as much as
> I've seen IPMI cards hang too.....
> 
> I think a remote controlled robot that can physically pull out power
> cables would be great!
> 
> Thanks
> 
> Alex

My personal approach is to use IPMI as the primary fence method with a
pair of switched PDUs (APC AP7900 model) as the backup method. This way,
when IPMI works I can be 100% sure it succeeded. However, as you say,
IPMI can fail for any number of reasons. In those cases, the PDUs
provide an alternative method, though it is up to you to ensure the
cables are in the right place as the PDU fence agent can only confirm
that the outlets have been opened, not that the node is off.

Here's how to do it:

http://clusterlabs.org/wiki/STONITH_Levels

-- 
Digimer
Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without
access to education?