<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body smarttemplateinserted="true" bgcolor="#FFFFFF" text="#000000">
<div id="smartTemplate4-quoteHeader">
<div><b>From: </b>Andrew Beekhof <a class="moz-txt-link-rfc2396E" href="mailto:andrew@beekhof.net"><andrew@beekhof.net></a></div>
<div><b>Sent: </b> 2014-06-20 04:48:25 EDT </div>
<div><b>To: </b>The Pacemaker cluster resource manager
<a class="moz-txt-link-rfc2396E" href="mailto:pacemaker@oss.clusterlabs.org"><pacemaker@oss.clusterlabs.org></a></div>
<div><b>Subject: </b>Re: [Pacemaker] Alternative communication
engine to corosync (etcd/consul/zookeeper/doozerd)</div>
<br>
</div>
<blockquote
cite="mid:6E9923E9-6708-47E3-8C80-233F2A68B7F8@beekhof.net"
type="cite">
<pre wrap="">
On 20 Jun 2014, at 2:14 pm, Patrick Hemmer <a class="moz-txt-link-rfc2396E" href="mailto:pacemaker@feystorm.net"><pacemaker@feystorm.net></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">After the demise of the old heartbeat service, and the switch to corosync as the primary (sole) method of communication between nodes,
</pre>
</blockquote>
<pre wrap="">
heartbeat is still supported as a messaging/membership layer
</pre>
<blockquote type="cite">
<pre wrap="">has there ever been any consideration into using services such as etcd, consul, zookeeper, or doozerd?
</pre>
</blockquote>
<pre wrap="">
I think most of these didn't exist at the time.
Do any provide membership too? What about message ordering?</pre>
</blockquote>
Yes. N/A (see below on the CPG vs KVS).<br>
<blockquote
cite="mid:6E9923E9-6708-47E3-8C80-233F2A68B7F8@beekhof.net"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">
These alternative communication engines offer some stuff that corosync doesn't. One such item etcd & consul offer is dynamic cluster scaling capabilities (nodes can very easily join and leave the cluster). When working in cloud computing, this feature becomes very important. Pcs is somewhat helpful in this regard but it's still nowhere near as capable (plus corosync doesn't have downscaling finished).
However one critical difference between these services and corosync is that they are mainly key/value stores, and don't have something like Corosync's CPG.
</pre>
</blockquote>
<pre wrap="">
Yes, a rather critical difference.
</pre>
<blockquote type="cite">
<pre wrap="">Though you could probably implement something looking like CPG using a keyvalue store,
</pre>
</blockquote>
<pre wrap="">
I rather doubt it. k/v stores and CPG are not very alike from where I'm sitting.</pre>
</blockquote>
No, they are not alike, but you could implement something looking
like CPG.<br>
When a key is created, that's a CPG message. They support atomic
operations (collision checking), so no 2 nodes could update a key at
the same time. So while it's a different underlying system, the end
result is the same, ordered messages.<br>
<blockquote
cite="mid:6E9923E9-6708-47E3-8C80-233F2A68B7F8@beekhof.net"
type="cite">
<blockquote type="cite">
<pre wrap="">I think pacemaker might be able to use a key/value store natively.</pre>
</blockquote>
</blockquote>
But I wouldn't even bother with hacking a KVS into something like
CPG if it's not needed. I would do it such that the CIB is stored as
keys and values natively. I would even think this is more efficient.
I'm not sure how the CIB is transmitted between nodes, but I think
it easiest to just set a single key when you want to update
something like a resource's last-rc-change value.<br>
<br>
<blockquote
cite="mid:6E9923E9-6708-47E3-8C80-233F2A68B7F8@beekhof.net"
type="cite">
<blockquote type="cite">
<pre wrap="">
So, has this ever been considered? How heavily tied is pacemaker to the corosync API? Could that be abstracted out enough to where different communication engines could be implemented?
</pre>
</blockquote>
<pre wrap="">
It's already abstracted to the point where it supports 4 different messaging/membership options:
- heartbeat
- corosync/pacemaker plugin
- cman
- corosync 2</pre>
</blockquote>
But aside from heartbeat, they all use corosync underneath. I wasn't
aware heartbeat was still supported. I assumed it was dropped. Bad
assumption :-)<br>
<br>
<blockquote
cite="mid:6E9923E9-6708-47E3-8C80-233F2A68B7F8@beekhof.net"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">
-Patrick
_______________________________________________
Pacemaker mailing list: <a class="moz-txt-link-abbreviated" href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a>
<a class="moz-txt-link-freetext" href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a>
Project Home: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org">http://www.clusterlabs.org</a>
Getting started: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a>
Bugs: <a class="moz-txt-link-freetext" href="http://bugs.clusterlabs.org">http://bugs.clusterlabs.org</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Pacemaker mailing list: <a class="moz-txt-link-abbreviated" href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a>
<a class="moz-txt-link-freetext" href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a>
Project Home: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org">http://www.clusterlabs.org</a>
Getting started: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a>
Bugs: <a class="moz-txt-link-freetext" href="http://bugs.clusterlabs.org">http://bugs.clusterlabs.org</a>
</pre>
</blockquote>
<br>
</body>
</html>