<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <div class="moz-cite-prefix">On 7/20/20 11:09 AM, Andrei Borzenkov
      wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAA91j0UunXD_GmmLZUjon_E5d8vFxE_6ginucAYgfSGxn1BGkQ@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <div dir="ltr">
        <div dir="ltr">
          <div class="gmail_default"
            style="font-family:arial,sans-serif"><br>
          </div>
        </div>
        <br>
        <div class="gmail_quote">
          <div dir="ltr" class="gmail_attr">On Mon, Jul 20, 2020 at
            11:45 AM Klaus Wenninger <<a
              href="mailto:kwenning@redhat.com" moz-do-not-send="true">kwenning@redhat.com</a>>
            wrote:<br>
          </div>
          <blockquote class="gmail_quote" style="margin:0px 0px 0px
            0.8ex;border-left:1px solid
            rgb(204,204,204);padding-left:1ex">
            <div>
              <div>On 7/20/20 10:34 AM, Andrei Borzenkov wrote:<br>
              </div>
              <blockquote type="cite">
                <div dir="ltr">
                  <div dir="ltr">
                    <div style="font-family:arial,sans-serif"><br>
                    </div>
                  </div>
                  <br>
                  <div class="gmail_quote"> 
                    <blockquote class="gmail_quote" style="margin:0px
                      0px 0px 0.8ex;border-left:1px solid
                      rgb(204,204,204);padding-left:1ex"> The
                      cpg-configuration sounds interesting as well.
                      Haven't used<br>
                      it or looked into the details. Would be interested
                      to hear about<br>
                      how that works.<br>
                    </blockquote>
                    <div><br>
                    </div>
                    <div>
                      <div style="font-family:arial,sans-serif">It
                        maintains a registry of VM location (each
                        fence_virtd polls local hypervisor at regular
                        intervals) and forwards fencing request to
                        appropriate host via corosync interconnect. It
                        is also the only backend that can handle host
                        failure - if it is known that host left cluster,
                        any VM on this host is considered fenced by
                        definition.</div>
                      <div style="font-family:arial,sans-serif"><br>
                      </div>
                      <div style="font-family:arial,sans-serif">It
                        requires that hosts are configured in pacemaker
                        cluster themselves (to handle host outage it
                        must be properly fenced).<br>
                      </div>
                    </div>
                  </div>
                </div>
              </blockquote>
              That sounds definitely interesting.<br>
              Are you saying that the hosts have to be pacemaker-nodes
              as well?<br>
            </div>
          </blockquote>
          <div><br>
          </div>
          <div>
            <div style="font-family:arial,sans-serif"
              class="gmail_default">cpg backend requires a working
              corosync cluster (it is using it as transport). It
              responds to "node joined" and "node left" events. So the
              question is when "node left" is generated. My
              understanding so far was that for unavailable node to be
              considered "left cluster" node must be fenced. If I am
              wrong, pacemaker is not needed. If fencing is required, I
              am not aware how it can be implemented without pacemaker.</div>
          </div>
        </div>
      </div>
    </blockquote>
    Node joined / left from corosync perspective doesn't require any
    fencing.<br>
    cpg is just sitting on top of corosync and doesn't know about
    pacemaker, fencing ...<br>
    and you can run multiple applications using multiple cpg-protocols
    on top of<br>
    a single corosync-instance. If this is useful of course depends on
    your scenario,<br>
    the number of tenants involved, ...<br>
    Anyway I probably got it wrong. Thought it would as well use the
    cpg-protocol<br>
    as transport between the fence-agents and the service instances.
    Which seems<br>
    not to be the case but would have caused that issue with multiple
    corosync<br>
    instances running on a single host.<br>
    <blockquote type="cite"
cite="mid:CAA91j0UunXD_GmmLZUjon_E5d8vFxE_6ginucAYgfSGxn1BGkQ@mail.gmail.com">
      <div dir="ltr">
        <div class="gmail_quote">
          <div>
            <div style="font-family:arial,sans-serif"
              class="gmail_default"><br>
            </div>
            <div style="font-family:arial,sans-serif"
              class="gmail_default">In any case, it is a completely
              separate cluster, so "as well" is not applicable.<br>
            </div>
            <br>
          </div>
          <div> </div>
          <blockquote class="gmail_quote" style="margin:0px 0px 0px
            0.8ex;border-left:1px solid
            rgb(204,204,204);padding-left:1ex">
            <div> Otherwise we might be able to just add them to
              corosync and configure<br>
              them not to vote on quorum ...<br>
            </div>
          </blockquote>
          <div><br>
          </div>
          <div>
            <div style="font-family:arial,sans-serif"
              class="gmail_default">These clusters are on different
              levels. Consider multi-tenant deployment. Each VM cluster
              has separate owner, and is managed independently. There is
              no reason to integrate it into underlying host cluster.
              Host cluster is managed by provider, not by tenant.<br>
            </div>
            <br>
          </div>
          <div> </div>
          <blockquote class="gmail_quote" style="margin:0px 0px 0px
            0.8ex;border-left:1px solid
            rgb(204,204,204);padding-left:1ex">
            <div> ... the same knet might then even be used to connect
              the bridges<br>
              on the hosts with each other on layer-2 ...<br>
            </div>
          </blockquote>
          <div><br>
          </div>
          <div>
            <div style="font-family:arial,sans-serif"
              class="gmail_default">With distributed backend you do not
              need any network connectivity between host and guest at
              all - just contact local hypervisor via local channel.
              That is actually more secure.</div>
          </div>
        </div>
      </div>
    </blockquote>
    That was referring to a simple setup where you have a bridge on each<br>
    host where the VM interfaces  are enslaved to and you don't need any<br>
    connectivity of the nodes to the outside world via these interfaces
    but you<br>
    would like to see the VMs behind interfaces enslaved to bridges on<br>
    other hosts.<br>
    So you could use a single knet for the corosync traffic, tunneling
    traffic<br>
    between hosts and potentially for fencing requests. Everything the<br>
    infrastructure below sees is this one knet.<br>
    Probably not suitable for a scenario with a lot of tenants but easy<br>
    if you want to quickly spin up a test cluster on a couple of hosts<br>
    without the need of any administration outside these hosts.
    <blockquote type="cite"
cite="mid:CAA91j0UunXD_GmmLZUjon_E5d8vFxE_6ginucAYgfSGxn1BGkQ@mail.gmail.com">
      <div dir="ltr">
        <div class="gmail_quote">
          <div><br>
          </div>
          <div> </div>
          <blockquote class="gmail_quote" style="margin:0px 0px 0px
            0.8ex;border-left:1px solid
            rgb(204,204,204);padding-left:1ex">
            <div><br>
            </div>
          </blockquote>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>