[ClusterLabs] Live migration not working on shutdown

[IT Nerb GmbH]

Zitat von Klaus Wenninger <kwenning at redhat.com>:

> On 11/02/2016 06:32 PM, Ken Gaillot wrote:
>> On 10/26/2016 06:12 AM, Rainer Nerb wrote:
>>> Hello all,
>>>
>>> we're currently testing a 2-node-cluster with 2 vms and live migration
>>> on CentOS 7.2 and Pacemaker 1.1.13-10 with disks on iSCSI-targets and
>>> migration via ssh-method.
>>>
>>> Live migration works, if we issue "pcs resource move ...", "pcs cluster
>>> standby", "pcs cluster stop" and even "systemctl rescue".
>>> The latter only worked, after adding the following additional
>>> dependencies to pacemaker.service and leaving the management of those
>>> services to systemd:
>>>
>>>   * After/Requires=systemd-machined.service
>>>   * After/Requires=systemd-machine-id-commit.service
>>>   * After/Requires=remote-fs.target
>>>   * After/Requires=libvirtd.service
>>>   * After/Requires=iscsi.service
>>>   * After/Requires=iscsid.service
>>>   * After/Requires=sshd.service
>> This makes sense when clustered resources depend on services that aren't
>> themselves managed by the cluster. It's dependent on your situation, so
>> it's not something that pacemaker can solve generically.
First approach was to use systemd-resources as there are no ocf:  
resource-agents for iSCSI-Initiators or libvirtd in our distribution.  
But then migration failed even on "systemctl rescue".
>>
>> You may already be aware, but the easiest way to add such requirements
>> is to put them in a systemd unit override, e.g.
>> /etc/systemd/system/pacemaker.service.d/dependencies.conf.
Yes, that's how we implemented the additional dependencies.
>>
>>> When shutting down or rebooting migration fails and not even the regular
>>> shutdown of the vms succeeds. Systemd seems to tear down the vms by
>>> terminating something they depend on.
>>>
>>> Is this a known issue? Did we miss any further dependencies?
>> There was a shutdown issue when using systemd-class cluster resources
>> (systemd: instead of ocf:), but I believe that was fixed in the package
>> you're using, and it's probably not relevant here anyway.
> Speaking of
> https://github.com/ClusterLabs/pacemaker/pull/887/commits/6aae8542abedc755b90c8c49aa5c429718fd12f1?
>
> It shouldn't be in Centos 7.2 but I agree unless there are no
> systemd-resources involved it wouldn't matter.
>
>>
>> It does sound like there's another dependency, but I don't know what.
>>
>> What log messages do you see on the failure?
See attached log files.
>>
>>> Tia
>>> Rainer
>>>
>>>
>>> ------------------------------------------------------------------------
>>> IT Nerb GmbH
>>> Lessingstraße 8
>>> 85098 Großmehring
>>>
>>> Telefon 	: 	+49 700 ITNERBGMBH
>>> Telefax 	: 	+49 8407 939 284
>>> email 	: 	info at it-nerb.de
>>> Internet 	: 	www.it-nerb.de <http://www.it-nerb.de>
>>> Geschäftsführer	:	Rainer Nerb
>>> Handelsregister	:	HRB 2592
>>> HR-Gericht	:	Ingolstadt
>>>
>>> ------------------------------------------------------------------------
>> _______________________________________________
>> Users mailing list: Users at clusterlabs.org
>> http://clusterlabs.org/mailman/listinfo/users
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>
>
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org




-----------------------------------------------------------------------
IT Nerb GmbH
Lessingstraße 8
85098 Großmehring

Telefon        : +49 700 ITNERBGMBH
Telefax        : +49 8407 939 284
email          : info at it-nerb.de
Internet       : www.it-nerb.de

Geschäftsführer: Rainer Nerb
Handelsregister: HRB 2592
HR-Gericht     : Ingolstadt
-----------------------------------------------------------------------
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: failed migration after "systemctl poweroff" - corosync.log
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20161104/62c49e28/attachment-0002.log>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: successful migration after "systemctl rescue" - corosync.log
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20161104/62c49e28/attachment-0003.log>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: failed migration - messages
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20161104/62c49e28/attachment-0007.ksh>