[ClusterLabs] fence_vmware_soap: fail to shutdown VMs
Klaus Wenninger
kwenning at redhat.com
Mon Jul 11 11:10:25 UTC 2016
On 07/11/2016 12:35 PM, Marek Grac wrote:
> Hi,
>
> 90MB of logs are not a big deal, most of them will just attempt to do
> same request again and again. Feel, free to send me a link to this file.
>
> If you have python-suds then it should be enough, you may try a
> different version of this package but we don't have any additional 3rd
> party dependencies afaik.
>
> m,
>
> On Mon, Jul 4, 2016 at 11:25 AM, Kevin THIERRY
> <kevin.thierry.citlao at gmail.com
> <mailto:kevin.thierry.citlao at gmail.com>> wrote:
>
> Thanks a lot for your reply Marek.
>
> Both fence-agents-common and fence-agents-vmware-soap are at
> version 4.0.11-27.
>
> I tried to add --power-timeout but it doesn't matter how long I
> set the power timeout, it always fails after about 4 seconds.
>
Guess this timeout just applies if it already had contact and
successfully could issue the command to ESX that turns off the power.
The command then is probably allowed to take that long.
In your case there seem to be more basic issues I guess which prevent
the contact in the first place.
Not having any experience with the vmware-fence-agents I'm guessing into
the blue but do
you maybe have different naming for the nodes seen via
pacemaker/corosync and as vmware-guests?
In this case (at least as I know it from fence_xvm) you probably have to
add some mapping so that
the stonith infrastructure knows which vmware-guest to fence when it
wants to fence a certain
pacemaker-node. (pcmk_host_map=node1:vmware-guest-node1;...)
Regards,
Klaus
> If I add -v I end up with *a lot* of output (~93MB) which mostly
> consist of xml. I am thinking this is not the kind of output that
> should be expected. Anyway I tried to look for the name of my VM
> in the logs but it doesn't even appear once.
>
> Here are the first 50 lines of the logs:
>
> ##############################################
>
> # head -n 50 fence-vmware-log.xml
> Delay 0 second(s) before logging in to the fence device
> reading wsdl at: https://10.5.200.20:443/sdk/vimService.wsdl ...
> opening (https://10.5.200.20:443/sdk/vimService.wsdl)
> <?xml version="1.0" encoding="UTF-8" ?>
> <!--
> Copyright 2005-2015 VMware, Inc. All rights reserved.
> -->
> <definitions targetNamespace="urn:vim25Service"
> xmlns="http://schemas.xmlsoap.org/wsdl/"
> <http://schemas.xmlsoap.org/wsdl/>
> xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
> <http://schemas.xmlsoap.org/wsdl/soap/>
> xmlns:interface="urn:vim25"
> >
> <import location="vim.wsdl" namespace="urn:vim25" />
> <service name="VimService">
> <port binding="interface:VimBinding" name="VimPort">
> <soap:address location="https://localhost/sdk/vimService"
> <https://localhost/sdk/vimService> />
> </port>
> </service>
> </definitions>
>
> sax duration: 1 (ms)
> warning: tns (urn:vim25Service), not mapped to prefix
> importing (vim.wsdl)
> reading wsdl at: https://10.5.200.20:443/sdk/vim.wsdl ...
> opening (https://10.5.200.20:443/sdk/vim.wsdl)
> <?xml version="1.0" encoding="UTF-8" ?>
> <!--
> Copyright 2005-2015 VMware, Inc. All rights reserved.
> -->
> <definitions targetNamespace="urn:vim25"
> xmlns="http://schemas.xmlsoap.org/wsdl/"
> <http://schemas.xmlsoap.org/wsdl/>
> xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/"
> <http://schemas.xmlsoap.org/wsdl/mime/>
> xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
> <http://schemas.xmlsoap.org/wsdl/soap/>
> xmlns:vim25="urn:vim25"
> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
> <http://www.w3.org/2001/XMLSchema>
> >
> <types>
> <schema
> targetNamespace="urn:vim25"
> xmlns="http://www.w3.org/2001/XMLSchema"
> <http://www.w3.org/2001/XMLSchema>
> xmlns:vim25="urn:vim25"
> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
> <http://www.w3.org/2001/XMLSchema>
> xmlns:reflect="urn:reflect"
> elementFormDefault="qualified"
> >
> <include schemaLocation="query-messagetypes.xsd" />
> <include schemaLocation="core-types.xsd" />
> <import namespace="urn:reflect"
> schemaLocation="reflect-messagetypes.xsd" />
> <include schemaLocation="vim-messagetypes.xsd" />
> <element name="versionURI" type="xsd:string" />
>
> ##############################################
>
> With -v, the error I get at the end of the logs is: "Unable to
> connect/login to fencing device" which is weird since I can get
> the status of a VM without issue...
>
> Could it be something I forgot to install on my machine (a library
> or something else)? I also thought about permissions issues but I
> am using the default root user and I can shutdown VM through
> vSphere with it.
>
> Ideas about that issue are more than welcome :)
>
> Kevin
>
>
> On 07/04/2016 02:09 PM, Marek Grac wrote:
>> Hi,
>>
>> you can try to raise value of --power-timeout from default (20
>> seconds), also you can add -v to have verbose output.
>>
>> As long as you have same version of fence-agents-common and
>> fence-agents-vmware, there should be no issues.
>>
>> m,
>>
>>
>> On Fri, Jul 1, 2016 at 11:31 AM, Kevin THIERRY
>> <kevin.thierry.citlao at gmail.com
>> <mailto:kevin.thierry.citlao at gmail.com>> wrote:
>>
>> Hello !
>>
>> I'm trying to fence my nodes using fence_vmware_soap but it
>> fails to shutdown or reboot my VMs. I can get the list of the
>> VMs on a host or query the status of a specific VM without
>> problem:
>>
>> # fence_vmware_soap -a 10.5.200.20 -l root -p "******" -z
>> --ssl-insecure -4 -n laa-billing-backup -o status
>> /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769:
>> InsecureRequestWarning:
>> Unverified HTTPS request is being made. Adding certificate
>> verification is strongly advised. See:
>> https://urllib3.readthedocs.org/en/latest/security.html
>> InsecureRequestWarning)
>> Status: ON
>>
>> However, trying to shutdown or to reboot a VM fails:
>>
>> # fence_vmware_soap -a 10.5.200.20 -l root -p "******" -z
>> --ssl-insecure -4 -n laa-billing-backup -o reboot
>> /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769:
>> InsecureRequestWarning: Unverified HTTPS request is being
>> made. Adding certificate verification is strongly advised.
>> See: https://urllib3.readthedocs.org/en/latest/security.html
>> InsecureRequestWarning)
>> Failed: Timed out waiting to power OFF
>>
>> On the ESXi I get the following logs in /var/log/hostd.log:
>>
>> [LikewiseGetDomainJoinInfo:355] QueryInformation():
>> ERROR_FILE_NOT_FOUND (2/0):
>> Accepted password for user root from 10.5.200.12
>> 2016-07-01T08:49:50.911Z info hostd[34380B70]
>> [Originator at 6876 sub=Vimsvc.ha-eventmgr opID=47defdf1] Event
>> 190 : User root at 10.5.200.12 <mailto:root at 10.5.200.12> logged
>> in as python-requests/2.6.0 CPython/2.7.5
>> Linux/3.10.0-327.18.2.el7.x86_64
>> 2016-07-01T08:49:50.998Z info hostd[32F80B70]
>> [Originator at 6876 sub=Vimsvc.TaskManager opID=47defdf4
>> user=root] Task Created : haTask--vim.SearchIndex.findByUuid-2513
>> 2016-07-01T08:49:50.999Z info hostd[32F80B70]
>> [Originator at 6876 sub=Vimsvc.TaskManager opID=47defdf4
>> user=root] Task Completed :
>> haTask--vim.SearchIndex.findByUuid-2513 Status success
>> 2016-07-01T08:49:51.009Z info hostd[32F80B70]
>> [Originator at 6876 sub=Solo.Vmomi opID=47defdf6 user=root]
>> Activation [N5Vmomi10ActivationE:0x34603c28] : Invoke done
>> [powerOff] on [vim.VirtualMachine:3]
>> 2016-07-01T08:49:51.009Z info hostd[32F80B70]
>> [Originator at 6876 sub=Solo.Vmomi opID=47defdf6 user=root]
>> Throw vim.fault.RestrictedVersion
>> 2016-07-01T08:49:51.009Z info hostd[32F80B70]
>> [Originator at 6876 sub=Solo.Vmomi opID=47defdf6 user=root] Result:
>> --> (vim.fault.RestrictedVersion) {
>> --> faultCause = (vmodl.MethodFault) null,
>> --> msg = ""
>> --> }
>> 2016-07-01T08:49:51.027Z info hostd[34380B70]
>> [Originator at 6876 sub=Vimsvc.ha-eventmgr opID=47defdf7
>> user=root] Event 191 : User root at 10.5.200.12
>> <mailto:root at 10.5.200.12> logged out (login time: Friday, 01
>> July, 2016 08:49:50, number of API invocations: 0, user
>> agent: python-requests/2.6.0 CPython/2.7.5
>> Linux/3.10.0-327.18.2.el7.x86_64)
>>
>>
>> I am wondering if there is some kind of compatibility issue.
>> I am using fence-agents-vmware-soap 4.0.11 on CentOS 7.2.1511
>> and ESXi 6.0.0 Build 2494585.
>> Any ideas about that issue?
>>
>> Best regards,
>>
>> --
>> Kevin THIERRY
>> IT System Engineer
>>
>> CIT Lao Ltd. – A.T.M.
>> PO Box 10082
>> Vientiane Capital – Lao P.D.R.
>> Cell : +856 (0)20 2221 8623
>> kevin.thierry.citlao at gmail.com
>> <mailto:kevin.thierry.citlao at gmail.com>
>>
>>
>> _______________________________________________
>> Users mailing list: Users at clusterlabs.org
>> <mailto:Users at clusterlabs.org>
>> http://clusterlabs.org/mailman/listinfo/users
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started:
>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>>
>>
>>
>>
>> _______________________________________________
>> Users mailing list: Users at clusterlabs.org <mailto:Users at clusterlabs.org>
>> http://clusterlabs.org/mailman/listinfo/users
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> <mailto:Users at clusterlabs.org>
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started:
> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
>
>
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
More information about the Users
mailing list