[ClusterLabs] fence_vmware_soap: fail to shutdown VMs
Kevin THIERRY
kevin.thierry.citlao at gmail.com
Mon Jul 11 22:02:48 EDT 2016
Hello,
Thanks a lot for your replies :)
For now I am fencing my two nodes with fence_ipmilan since each of my
two nodes is running alone on its own physical server. It works well but
I am still interested in fencing them with fence_vmware_soap.
On 07/11/2016 06:10 PM, Klaus Wenninger wrote:
> On 07/11/2016 12:35 PM, Marek Grac wrote:
>> Hi,
>>
>> 90MB of logs are not a big deal, most of them will just attempt to do
>> same request again and again. Feel, free to send me a link to this file.
Here is a link to the log file:
https://drive.google.com/file/d/0B6P7I1AC43u5bW5xNnNyR2dVWTA/view?usp=sharing
The command I executed is:
fence_vmware_soap -a 10.5.200.20 -l root -p "**********" -z
--ssl-insecure -n laa-billing-backup -o off
>>
>> If you have python-suds then it should be enough, you may try a
>> different version of this package but we don't have any additional 3rd
>> party dependencies afaik.
I have python-suds version 0.4.1 installed which seems to be the last
version available. Maybe I could try one of the more recent versions of
this fork: https://bitbucket.org/jurko/suds
>>
>> m,
>>
>> On Mon, Jul 4, 2016 at 11:25 AM, Kevin THIERRY
>> <kevin.thierry.citlao at gmail.com
>> <mailto:kevin.thierry.citlao at gmail.com>> wrote:
>>
>> Thanks a lot for your reply Marek.
>>
>> Both fence-agents-common and fence-agents-vmware-soap are at
>> version 4.0.11-27.
>>
>> I tried to add --power-timeout but it doesn't matter how long I
>> set the power timeout, it always fails after about 4 seconds.
>>
> Guess this timeout just applies if it already had contact and
> successfully could issue the command to ESX that turns off the power.
> The command then is probably allowed to take that long.
> In your case there seem to be more basic issues I guess which prevent
> the contact in the first place.
That makes sense but I still have no idea about what is wrong. It is
possible that the error is a misconfiguration on the ESXi side but I'm
not sure what to look at since I didn't change many things from the
default configuration.
> Not having any experience with the vmware-fence-agents I'm guessing into
> the blue but do
> you maybe have different naming for the nodes seen via
> pacemaker/corosync and as vmware-guests?
> In this case (at least as I know it from fence_xvm) you probably have to
> add some mapping so that
> the stonith infrastructure knows which vmware-guest to fence when it
> wants to fence a certain
> pacemaker-node. (pcmk_host_map=node1:vmware-guest-node1;...)
Right now I am just executing fence_vmware_soap manually but you are
right and that is what I did to fence the server using fence_ipmilan.
For fence_vmware_soap I also tried to use the VM UUID instead of its
name but I got the exact same result. Also when I just check the VM
status or try to get the VM list on an ESXi it works fine.
>
> Regards,
> Klaus
>
>> If I add -v I end up with *a lot* of output (~93MB) which mostly
>> consist of xml. I am thinking this is not the kind of output that
>> should be expected. Anyway I tried to look for the name of my VM
>> in the logs but it doesn't even appear once.
>>
>> Here are the first 50 lines of the logs:
>>
>> ##############################################
>>
>> # head -n 50 fence-vmware-log.xml
>> Delay 0 second(s) before logging in to the fence device
>> reading wsdl at: https://10.5.200.20:443/sdk/vimService.wsdl ...
>> opening (https://10.5.200.20:443/sdk/vimService.wsdl)
>> <?xml version="1.0" encoding="UTF-8" ?>
>> <!--
>> Copyright 2005-2015 VMware, Inc. All rights reserved.
>> -->
>> <definitions targetNamespace="urn:vim25Service"
>> xmlns="http://schemas.xmlsoap.org/wsdl/"
>> <http://schemas.xmlsoap.org/wsdl/>
>> xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
>> <http://schemas.xmlsoap.org/wsdl/soap/>
>> xmlns:interface="urn:vim25"
>> >
>> <import location="vim.wsdl" namespace="urn:vim25" />
>> <service name="VimService">
>> <port binding="interface:VimBinding" name="VimPort">
>> <soap:address location="https://localhost/sdk/vimService"
>> <https://localhost/sdk/vimService> />
>> </port>
>> </service>
>> </definitions>
>>
>> sax duration: 1 (ms)
>> warning: tns (urn:vim25Service), not mapped to prefix
>> importing (vim.wsdl)
>> reading wsdl at: https://10.5.200.20:443/sdk/vim.wsdl ...
>> opening (https://10.5.200.20:443/sdk/vim.wsdl)
>> <?xml version="1.0" encoding="UTF-8" ?>
>> <!--
>> Copyright 2005-2015 VMware, Inc. All rights reserved.
>> -->
>> <definitions targetNamespace="urn:vim25"
>> xmlns="http://schemas.xmlsoap.org/wsdl/"
>> <http://schemas.xmlsoap.org/wsdl/>
>> xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/"
>> <http://schemas.xmlsoap.org/wsdl/mime/>
>> xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
>> <http://schemas.xmlsoap.org/wsdl/soap/>
>> xmlns:vim25="urn:vim25"
>> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
>> <http://www.w3.org/2001/XMLSchema>
>> >
>> <types>
>> <schema
>> targetNamespace="urn:vim25"
>> xmlns="http://www.w3.org/2001/XMLSchema"
>> <http://www.w3.org/2001/XMLSchema>
>> xmlns:vim25="urn:vim25"
>> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
>> <http://www.w3.org/2001/XMLSchema>
>> xmlns:reflect="urn:reflect"
>> elementFormDefault="qualified"
>> >
>> <include schemaLocation="query-messagetypes.xsd" />
>> <include schemaLocation="core-types.xsd" />
>> <import namespace="urn:reflect"
>> schemaLocation="reflect-messagetypes.xsd" />
>> <include schemaLocation="vim-messagetypes.xsd" />
>> <element name="versionURI" type="xsd:string" />
>>
>> ##############################################
>>
>> With -v, the error I get at the end of the logs is: "Unable to
>> connect/login to fencing device" which is weird since I can get
>> the status of a VM without issue...
>>
>> Could it be something I forgot to install on my machine (a library
>> or something else)? I also thought about permissions issues but I
>> am using the default root user and I can shutdown VM through
>> vSphere with it.
>>
>> Ideas about that issue are more than welcome :)
>>
>> Kevin
>>
>>
>> On 07/04/2016 02:09 PM, Marek Grac wrote:
>>> Hi,
>>>
>>> you can try to raise value of --power-timeout from default (20
>>> seconds), also you can add -v to have verbose output.
>>>
>>> As long as you have same version of fence-agents-common and
>>> fence-agents-vmware, there should be no issues.
>>>
>>> m,
>>>
>>>
>>> On Fri, Jul 1, 2016 at 11:31 AM, Kevin THIERRY
>>> <kevin.thierry.citlao at gmail.com
>>> <mailto:kevin.thierry.citlao at gmail.com>> wrote:
>>>
>>> Hello !
>>>
>>> I'm trying to fence my nodes using fence_vmware_soap but it
>>> fails to shutdown or reboot my VMs. I can get the list of the
>>> VMs on a host or query the status of a specific VM without
>>> problem:
>>>
>>> # fence_vmware_soap -a 10.5.200.20 -l root -p "******" -z
>>> --ssl-insecure -4 -n laa-billing-backup -o status
>>> /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769:
>>> InsecureRequestWarning:
>>> Unverified HTTPS request is being made. Adding certificate
>>> verification is strongly advised. See:
>>> https://urllib3.readthedocs.org/en/latest/security.html
>>> InsecureRequestWarning)
>>> Status: ON
>>>
>>> However, trying to shutdown or to reboot a VM fails:
>>>
>>> # fence_vmware_soap -a 10.5.200.20 -l root -p "******" -z
>>> --ssl-insecure -4 -n laa-billing-backup -o reboot
>>> /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769:
>>> InsecureRequestWarning: Unverified HTTPS request is being
>>> made. Adding certificate verification is strongly advised.
>>> See: https://urllib3.readthedocs.org/en/latest/security.html
>>> InsecureRequestWarning)
>>> Failed: Timed out waiting to power OFF
>>>
>>> On the ESXi I get the following logs in /var/log/hostd.log:
>>>
>>> [LikewiseGetDomainJoinInfo:355] QueryInformation():
>>> ERROR_FILE_NOT_FOUND (2/0):
>>> Accepted password for user root from 10.5.200.12
>>> 2016-07-01T08:49:50.911Z info hostd[34380B70]
>>> [Originator at 6876 sub=Vimsvc.ha-eventmgr opID=47defdf1] Event
>>> 190 : User root at 10.5.200.12 <mailto:root at 10.5.200.12> logged
>>> in as python-requests/2.6.0 CPython/2.7.5
>>> Linux/3.10.0-327.18.2.el7.x86_64
>>> 2016-07-01T08:49:50.998Z info hostd[32F80B70]
>>> [Originator at 6876 sub=Vimsvc.TaskManager opID=47defdf4
>>> user=root] Task Created : haTask--vim.SearchIndex.findByUuid-2513
>>> 2016-07-01T08:49:50.999Z info hostd[32F80B70]
>>> [Originator at 6876 sub=Vimsvc.TaskManager opID=47defdf4
>>> user=root] Task Completed :
>>> haTask--vim.SearchIndex.findByUuid-2513 Status success
>>> 2016-07-01T08:49:51.009Z info hostd[32F80B70]
>>> [Originator at 6876 sub=Solo.Vmomi opID=47defdf6 user=root]
>>> Activation [N5Vmomi10ActivationE:0x34603c28] : Invoke done
>>> [powerOff] on [vim.VirtualMachine:3]
>>> 2016-07-01T08:49:51.009Z info hostd[32F80B70]
>>> [Originator at 6876 sub=Solo.Vmomi opID=47defdf6 user=root]
>>> Throw vim.fault.RestrictedVersion
>>> 2016-07-01T08:49:51.009Z info hostd[32F80B70]
>>> [Originator at 6876 sub=Solo.Vmomi opID=47defdf6 user=root] Result:
>>> --> (vim.fault.RestrictedVersion) {
>>> --> faultCause = (vmodl.MethodFault) null,
>>> --> msg = ""
>>> --> }
>>> 2016-07-01T08:49:51.027Z info hostd[34380B70]
>>> [Originator at 6876 sub=Vimsvc.ha-eventmgr opID=47defdf7
>>> user=root] Event 191 : User root at 10.5.200.12
>>> <mailto:root at 10.5.200.12> logged out (login time: Friday, 01
>>> July, 2016 08:49:50, number of API invocations: 0, user
>>> agent: python-requests/2.6.0 CPython/2.7.5
>>> Linux/3.10.0-327.18.2.el7.x86_64)
>>>
>>>
>>> I am wondering if there is some kind of compatibility issue.
>>> I am using fence-agents-vmware-soap 4.0.11 on CentOS 7.2.1511
>>> and ESXi 6.0.0 Build 2494585.
>>> Any ideas about that issue?
>>>
>>> Best regards,
>>>
>>> --
>>> Kevin THIERRY
>>> IT System Engineer
>>>
>>> CIT Lao Ltd. – A.T.M.
>>> PO Box 10082
>>> Vientiane Capital – Lao P.D.R.
>>> Cell : +856 (0)20 2221 8623
>>> kevin.thierry.citlao at gmail.com
>>> <mailto:kevin.thierry.citlao at gmail.com>
>>>
>>>
Best regards,
Kevin
More information about the Users
mailing list