<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">On 7/20/20 11:09 AM, Andrei Borzenkov
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAA91j0UunXD_GmmLZUjon_E5d8vFxE_6ginucAYgfSGxn1BGkQ@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default"
style="font-family:arial,sans-serif"><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Jul 20, 2020 at
11:45 AM Klaus Wenninger <<a
href="mailto:kwenning@redhat.com" moz-do-not-send="true">kwenning@redhat.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<div>On 7/20/20 10:34 AM, Andrei Borzenkov wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div style="font-family:arial,sans-serif"><br>
</div>
</div>
<br>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex"> The
cpg-configuration sounds interesting as well.
Haven't used<br>
it or looked into the details. Would be interested
to hear about<br>
how that works.<br>
</blockquote>
<div><br>
</div>
<div>
<div style="font-family:arial,sans-serif">It
maintains a registry of VM location (each
fence_virtd polls local hypervisor at regular
intervals) and forwards fencing request to
appropriate host via corosync interconnect. It
is also the only backend that can handle host
failure - if it is known that host left cluster,
any VM on this host is considered fenced by
definition.</div>
<div style="font-family:arial,sans-serif"><br>
</div>
<div style="font-family:arial,sans-serif">It
requires that hosts are configured in pacemaker
cluster themselves (to handle host outage it
must be properly fenced).<br>
</div>
</div>
</div>
</div>
</blockquote>
That sounds definitely interesting.<br>
Are you saying that the hosts have to be pacemaker-nodes
as well?<br>
</div>
</blockquote>
<div><br>
</div>
<div>
<div style="font-family:arial,sans-serif"
class="gmail_default">cpg backend requires a working
corosync cluster (it is using it as transport). It
responds to "node joined" and "node left" events. So the
question is when "node left" is generated. My
understanding so far was that for unavailable node to be
considered "left cluster" node must be fenced. If I am
wrong, pacemaker is not needed. If fencing is required, I
am not aware how it can be implemented without pacemaker.</div>
</div>
</div>
</div>
</blockquote>
Node joined / left from corosync perspective doesn't require any
fencing.<br>
cpg is just sitting on top of corosync and doesn't know about
pacemaker, fencing ...<br>
and you can run multiple applications using multiple cpg-protocols
on top of<br>
a single corosync-instance. If this is useful of course depends on
your scenario,<br>
the number of tenants involved, ...<br>
Anyway I probably got it wrong. Thought it would as well use the
cpg-protocol<br>
as transport between the fence-agents and the service instances.
Which seems<br>
not to be the case but would have caused that issue with multiple
corosync<br>
instances running on a single host.<br>
<blockquote type="cite"
cite="mid:CAA91j0UunXD_GmmLZUjon_E5d8vFxE_6ginucAYgfSGxn1BGkQ@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div>
<div style="font-family:arial,sans-serif"
class="gmail_default"><br>
</div>
<div style="font-family:arial,sans-serif"
class="gmail_default">In any case, it is a completely
separate cluster, so "as well" is not applicable.<br>
</div>
<br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div> Otherwise we might be able to just add them to
corosync and configure<br>
them not to vote on quorum ...<br>
</div>
</blockquote>
<div><br>
</div>
<div>
<div style="font-family:arial,sans-serif"
class="gmail_default">These clusters are on different
levels. Consider multi-tenant deployment. Each VM cluster
has separate owner, and is managed independently. There is
no reason to integrate it into underlying host cluster.
Host cluster is managed by provider, not by tenant.<br>
</div>
<br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div> ... the same knet might then even be used to connect
the bridges<br>
on the hosts with each other on layer-2 ...<br>
</div>
</blockquote>
<div><br>
</div>
<div>
<div style="font-family:arial,sans-serif"
class="gmail_default">With distributed backend you do not
need any network connectivity between host and guest at
all - just contact local hypervisor via local channel.
That is actually more secure.</div>
</div>
</div>
</div>
</blockquote>
That was referring to a simple setup where you have a bridge on each<br>
host where the VM interfaces are enslaved to and you don't need any<br>
connectivity of the nodes to the outside world via these interfaces
but you<br>
would like to see the VMs behind interfaces enslaved to bridges on<br>
other hosts.<br>
So you could use a single knet for the corosync traffic, tunneling
traffic<br>
between hosts and potentially for fencing requests. Everything the<br>
infrastructure below sees is this one knet.<br>
Probably not suitable for a scenario with a lot of tenants but easy<br>
if you want to quickly spin up a test cluster on a couple of hosts<br>
without the need of any administration outside these hosts.
<blockquote type="cite"
cite="mid:CAA91j0UunXD_GmmLZUjon_E5d8vFxE_6ginucAYgfSGxn1BGkQ@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div><br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</body>
</html>