[ClusterLabs] fence_azure_arm flooding syslog

Mon Feb 11 13:47:02 UTC 2019

We need this feature because shutdown / reboot takes too much time ( > 5
min) and network fencing fences the virtual machine much faster ( < 5 sec).
We finished all the required steps and network fencing works as expected but
I'm still confused about these errors in the log and the failure counts
showed by pcs ...

fence_azure_arm: Keyring cache token has failed: No recommended backend was
available. Install the keyrings.alt package if you want to use the
non-recommended backends. See README.rst for details.

stonith-ng[7789]: warning: fence_azure_arm[7896] stderr: [ 2019-02-11
13:41:19,178 WARNING: Keyring cache token has failed: No recommended backend
was available. Install the keyrings.alt package if you want to use the
non-recommended backends. See README.rst for details. ]

stonith-net_monitor_60000 on node02 'unknown error' (1): call=369,
status=Timed Out, exitreason='', last-rc-change='Mon Feb 11 10:38:02 2019',
queued=0ms, exec=20007ms

Thomas

-----Original Message-----
From: Users [mailto:users-bounces at clusterlabs.org] On Behalf Of Oyvind
Albrigtsen
Sent: Montag, 11. Februar 2019 13:58
To: Cluster Labs - All topics related to open-source clustering welcomed
<users at clusterlabs.org>
Subject: Re: [ClusterLabs] fence_azure_arm flooding syslog

Oh. Thanks for clarifying.

Network-fencing requires extra setup as explained if you run "pcs stonith
resource describe fence_azure_arm".

You probably dont need network-fencing, so you can just skip that part.

On 11/02/19 13:50 +0100, Thomas Berreis wrote:
>> It seems like the issue might be the space in "pcmk_host_list= node01".
>Sorry, this typo was only in my mail because I anonymized in- and output.
>
>The issue only occurs when I add the parameter "network-fencing=on".
>
>WARNING:msrestazure.azure_active_directory:Keyring cache token has failed:
>No recommended backend was available. Install the keyrings.alt package 
>if you want to use the non-recommended backends. See README.rst for
details.
>2019-02-11 12:29:37,079 WARNING: Keyring cache token has failed: No 
>recommended backend was available. Install the keyrings.alt package if 
>you want to use the non-recommended backends. See README.rst for details.
>
>
>Thomas
>
>-----Original Message-----
>From: Users [mailto:users-bounces at clusterlabs.org] On Behalf Of Oyvind 
>Albrigtsen
>Sent: Montag, 11. Februar 2019 10:38
>To: Cluster Labs - All topics related to open-source clustering 
>welcomed <users at clusterlabs.org>
>Subject: Re: [ClusterLabs] fence_azure_arm flooding syslog
>
>pcmk_host_list is used by Pacemaker to limit which hosts can be fenced 
>by the STONITH device, so you'll have to use the plug parameter for
testing.
>Pacemaker uses it internally when it fences a host, so you wouldnt use 
>plug= as part of the pcs stonith create command.
>
>It seems like the issue might be the space in "pcmk_host_list= node01".
>
>Oyvind
>
>On 10/02/19 14:43 +0100, Thomas Berreis wrote:
>>Hi Oyvind,
>>
>>Seems, option "pcmk_host_list" is not supported by fence_azure_arm.
>>Instead fence_azure_arm requires "plug" and however this is not 
>>supported
>by pcs.
>>What's the correct syntax to add such a stonith device with
>fence_azure_arm?
>>
>>[root ~]# fence_azure_arm
>>login=***
>>network-fencing=on
>>passwd=***
>>pcmk_host_list= node01
>>resourceGroup=***
>>retry_on=0
>>subscriptionId=***
>>tenantId=***
>>WARNING:root:Parse error: Ignoring unknown option 'pcmk_host_list= node01'
>>
>>ERROR:root:Failed: You have to enter plug number or machine 
>>identification
>>
>>2019-02-10 12:54:51,862 ERROR: Failed: You have to enter plug number 
>>or machine identification
>>
>>------------------------------
>>
>>[root ~]# fence_azure_arm
>>login=***
>>network-fencing=on
>>passwd=***
>>pcmk_host_list= node01
>>resourceGroup=***
>>retry_on=0
>>subscriptionId=***
>>tenantId=***
>>plug=node01
>>WARNING:root:Parse error: Ignoring unknown option 'pcmk_host_list= node01'
>>
>>WARNING:msrestazure.azure_active_directory:Keyring cache token has failed:
>>No recommended backend was available.
>>2019-02-10 12:56:30,377 WARNING: Keyring cache token has failed: No 
>>recommended backend was available. Install th 
>>WARNING:msrestazure.azure_active_directory:Keyring cache token has failed:
>>No recommended backend was available.
>>2019-02-10 12:56:30,729 WARNING: Keyring cache token has failed: No 
>>recommended backend was available. Install th
>>Success: Powered OFF
>>
>>------------------------------
>>
>>[root ~]# fence_azure_arm
>>login=***
>>network-fencing=on
>>passwd=***
>>plug=node01
>>resourceGroup=***
>>retry_on=0
>>subscriptionId=***
>>tenantId=***
>>
>>Success: Powered OFF
>>
>>------------------------------
>>
>>[root ~]# pcs stonith create stonith-net fence_azure_arm [...]
>>plug=node01 network-fencing=on
>>Error: invalid stonith option 'plug'
>>
>>[root ~]# fence_azure_arm -h
>>Usage:
>>        fence_azure_arm [options]
>>Options:
>>[...]
>>   -n, --plug=[id]                Physical plug number on device, UUID or
>>identification of machine
>>
>>
>>Thanks!
>>
>>Thomas
>>
>>-----Original Message-----
>>From: Users [mailto:users-bounces at clusterlabs.org] On Behalf Of Oyvind 
>>Albrigtsen
>>Sent: Freitag, 8. Februar 2019 14:16
>>To: Cluster Labs - All topics related to open-source clustering 
>>welcomed <users at clusterlabs.org>
>>Subject: Re: [ClusterLabs] fence_azure_arm flooding syslog
>>
>>Hi Thomas,
>>
>>It looks like it's trying to use credentials from keyring, which 
>>generates these error messages.
>>
>>Maybe there's a typo in one of the authentitcation parameters in your 
>>config. Can you post the output of "pcs stonith show --full"?
>>
>>You can try running fence_azure_arm without parameters on the command 
>>line and copy the parameters (one per line) to it, and add e.g.
>>action=status and end with Ctrl+D to see .
>>
>>
>>Oyvind
>>
>>On 08/02/19 14:02 +0100, Thomas Berreis wrote:
>>>Hi,
>>>
>>>
>>>
>>>I'm having a two-node cluster in Azure with fencing enabled via 
>>>module fence_azure_arm.
>>>
>>>The cluster management is flooding my syslog on both nodes with these
>>>messages:
>>>
>>>
>>>
>>>** fence_azure_arm: Keyring cache token has failed: No recommended 
>>>backend was available. Install the keyrings.alt package if you want 
>>>to use the non-recommended backends. See README.rst for details.
>>>
>>>
>>>
>>>** stonith-ng[5220]: warning: fence_azure_arm[14847] stderr: [
>>>2019-02-08
>>>12:52:02,703 WARNING: Keyring cache token has failed: No recommended 
>>>backend was available. Install the keyrings.alt package if you want 
>>>to use the non-recommended backends. See README.rst for details. ]
>>>
>>>
>>>
>>>
>>>
>>>Can anyone give me a hint how to fix these issue?
>>>
>>>
>>>
>>>pcs version: 0.9.165
>>>
>>>pacemaker version: 1.1.19-8.el7_6.1
>>>
>>>fence_azure_arm version: 4.2.1
>>>
>>>corosync 2.4.3
>>>
>>>
>>>
>>>Thanks!
>>>
>>>
>>>
>>>Thomas
>>>
>>
>>>_______________________________________________
>>>Users mailing list: Users at clusterlabs.org 
>>>https://lists.clusterlabs.org/mailman/listinfo/users
>>>
>>>Project Home: http://www.clusterlabs.org Getting started:
>>>http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>Bugs: http://bugs.clusterlabs.org
>>
>>_______________________________________________
>>Users mailing list: Users at clusterlabs.org 
>>https://lists.clusterlabs.org/mailman/listinfo/users
>>
>>Project Home: http://www.clusterlabs.org Getting started:
>>http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>Bugs: http://bugs.clusterlabs.org
>>
>>_______________________________________________
>>Users mailing list: Users at clusterlabs.org 
>>https://lists.clusterlabs.org/mailman/listinfo/users
>>
>>Project Home: http://www.clusterlabs.org Getting started:
>>http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>Bugs: http://bugs.clusterlabs.org
>_______________________________________________
>Users mailing list: Users at clusterlabs.org 
>https://lists.clusterlabs.org/mailman/listinfo/users
>
>Project Home: http://www.clusterlabs.org Getting started:
>http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>Bugs: http://bugs.clusterlabs.org
>
>_______________________________________________
>Users mailing list: Users at clusterlabs.org 
>https://lists.clusterlabs.org/mailman/listinfo/users
>
>Project Home: http://www.clusterlabs.org Getting started: 
>http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>Bugs: http://bugs.clusterlabs.org
_______________________________________________
Users mailing list: Users at clusterlabs.org
https://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org Getting started:
http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org