[ClusterLabs] Live migrate a VM in a cluster group

[Jason Gauthier]

On Tue, May 29, 2018 at 11:46 AM, Ken Gaillot <kgaillot at redhat.com> wrote:
> On Tue, 2018-05-29 at 10:14 -0500, Ken Gaillot wrote:
>> On Sun, 2018-05-27 at 22:50 -0400, Jason Gauthier wrote:
>> > Greetings,
>> >
>> >  I've set up a cluster intended for VMs.  I created a VM, and have
>> > been pretty pleased with migrating it back and forth between the
>> > two
>> > nodes.  Now, I am also using the VNC console, which requires
>> > listening
>> > on port 59xx.  Of course, once the machine is migrated the IP to
>> > access the VNC console is different.
>> > So, I thought I would be clever and create a cluster IP address.  I
>> > did, and as a stand alone resource it migrates between nodes
>> > perfectly.  I then put these two primitives into a group.
>> >
>> > When I try to migrate the group nothing happens.
>> > There aren't any cluster errors, and the logs do not give me any
>> > kind
>> > of indication of error either.
>> >
>> > I'm wondering if this is even something I should expect to work. I
>> > would certainly like it to.
>> > Here's the relevant config:
>> >
>> > primitive p_Calibre VirtualDomain \
>> >         params config="/vms/Calibre/Calibre.xml"
>> > hypervisor="qemu:///system" migration_transport=ssh \
>> >         meta allow-migrate=true \
>> >         op start timeout=120s interval=0 \
>> >         op stop timeout=120s interval=0 \
>> >         op monitor timeout=30 interval=10 depth=0 \
>> >         utilization cpu=2 hv_memory=8192
>> > primitive p_CalibreVNC IPaddr2 \
>> >         params ip=192.168.10.10 cidr_netmask=24 nic=br10 \
>> >         op monitor interval=10s
>> >
>> > group g_Calibre p_CalibreVNC p_Calibre \
>> >         meta target-role=Started
>> >
>> > location cli-prefer-g_Calibre g_Calibre role=Started inf: alpha
>> > location cli-prefer-p_Calibre p_Calibre role=Started inf: beta
>>
>> The group prefers alpha, but its p_Calibre member prefers beta.
>> You're
>> confusing the cluster :)
>>
>> Any constraints starting with "cli-" were added by command-line tools
>> doing move/ban/etc. They stay in effect until they are manually
>> removed
>> (the same tool will generally have a "clear" option).
>
> Oh, and IPaddr2 can't live-migrate, so putting it in a group (or
> colocation+ordering constraint) with p_Calibre will make the entire
> group unable to live-migrate.
>
> What exact relationships do you really need?
>
> Does the IP need to be up before the VM starts in order for VNC to
> work? If so, can it even work with a live migration (the IP can't be up
> on both nodes at once)?
>
> If the IP doesn't need to be up first, you could simply colocate the IP
> with the VM, without any ordering constraints. Whenever the VM moves,
> the IP will follow; the VM would live-migrate, and the IP would
> stop+start.
>
> If the IP does need to be up first, I'm thinking you could create a
> group with p_CalibreVNC and an ocf:pacemaker:attribute resource (which
> sets a node attribute when it is running), and then use a rule-based
> constraint to colocate+order p_Calibre relative to the node attribute.
> I'd make the colocation non-infinite and the order optional, so the VM
> can stay up even if the IP fails or begins to move.
>
> You would trigger a migration by moving the group. It would proceed
> like this:
> - The IP, attribute resource, node attribute, and VM start out on node
> 1
> - The attribute resource stops on node 1, removing the node attribute
> from node 1 (this is where making the constraints optional comes in
> handy)
> - The IP resource stops on node 1
> - The IP resource starts on node 2
> - The attribute resource starts on node 2, adding the node attribute to
> node 2
> - The VM live-migrates to node 2
>
> That does leave a window where the IP and VM are on different nodes,
> but that sounds acceptable in your setup in order to preserve live
> migration.
>

You and Andrei were correct.  The contraints were causing an issue.  I
am able to move them as a unit, but as you said..
The results are unpleasant.  To achieve what I really want, I will
need to execute the plan you mentioned above.

I am not sure how to do that, but your steps are perfect.  I do not
care about the IP going up and down since it's just console
management.
But I do want the machine to stay up and be live migrated. So, I will
look into the process you mentioned.
It does seem that the IP needs to be up first, or the listener can't
bind, and qemu fails.

Thanks for the advice. I have some more learning to do.



>> > location cli-prefer-p_CalibreVNC p_CalibreVNC role=Started inf:
>> > beta
>> >
>> > Any help is appreciated.
> --
> Ken Gaillot <kgaillot at redhat.com>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> https://lists.clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org