<div dir="ltr">I tried to install corosync 3.x and it works pretty well.<div>But when I install pacemaker, it installs previous version of corosync as dependency and breaks all the setup.</div><div>Any suggestions?</div></div><div class="gmail_extra"><br><div class="gmail_quote">2018-08-23 9:32 GMT+02:00 Jan Friesse <span dir="ltr"><<a href="mailto:jfriesse@redhat.com" target="_blank">jfriesse@redhat.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">David,<span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
BTW, where I can download Corosync 3.x?<br>
I've only seen Corosync 2.99.3 Alpha4 at <a href="http://corosync.github.io/corosync/" rel="noreferrer" target="_blank">http://corosync.github.io/coro<wbr>sync/</a><br>
</blockquote>
<br></span>
Yes, that's Alpha 4 of Corosync 3.<div><div class="h5"><br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
2018-08-23 9:11 GMT+02:00 David Tolosa <<a href="mailto:david.tolosa@upcnet.es" target="_blank">david.tolosa@upcnet.es</a>>:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I'm currently using an Ubuntu 18.04 server configuration with netplan.<br>
<br>
Here you have my current YAML configuration:<br>
<br>
# This file describes the network interfaces available on your system<br>
# For more information, see netplan(5).<br>
network:<br>
version: 2<br>
renderer: networkd<br>
ethernets:<br>
eno1:<br>
addresses: [<a href="http://192.168.0.1/24" rel="noreferrer" target="_blank">192.168.0.1/24</a>]<br>
enp4s0f0:<br>
addresses: [<a href="http://192.168.1.1/24" rel="noreferrer" target="_blank">192.168.1.1/24</a>]<br>
enp5s0f0:<br>
{}<br>
vlans:<br>
vlan.XXX:<br>
id: XXX<br>
link: enp5s0f0<br>
addresses: [ <a href="http://10.1.128.5/29" rel="noreferrer" target="_blank">10.1.128.5/29</a> ]<br>
gateway4: 10.1.128.1<br>
nameservers:<br>
addresses: [ 8.8.8.8, 8.8.4.4 ]<br>
search: [ <a href="http://foo.com" rel="noreferrer" target="_blank">foo.com</a>, <a href="http://bar.com" rel="noreferrer" target="_blank">bar.com</a> ]<br>
vlan.YYY:<br>
id: YYY<br>
link: enp5s0f0<br>
addresses: [ <a href="http://10.1.128.5/29" rel="noreferrer" target="_blank">10.1.128.5/29</a> ]<br>
<br>
<br>
So, eno1 and enp4s0f0 are the two ethernet ports connected each other<br>
with crossover cables to node2.<br>
enp5s0f0 port is used to connect outside/services using vlans defined in<br>
the same file.<br>
<br>
In short, I'm using systemd-networkd default Ubuntu 18 server service for<br>
</blockquote></blockquote>
<br></div></div>
Ok, so systemd-networkd is really doing ifdown and somebody actually tries fix it and merge into upstream (sadly with not too much luck :( )<br>
<br>
<a href="https://github.com/systemd/systemd/pull/7403" rel="noreferrer" target="_blank">https://github.com/systemd/sys<wbr>temd/pull/7403</a><span class=""><br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
manage networks. Im not detecting any NetworkManager-config-server<br>
package in my repository neither.<br>
</blockquote></blockquote>
<br></span>
I'm not sure how it's called in Debian based distributions, but it's just one small file in /etc, so you can extract it from RPM.<span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
So the only solution that I have left, I suppose, is to test corosync 3.x<br>
and see if it works better handling RRP.<br>
</blockquote></blockquote>
<br></span>
You may also reconsider to try ether completely static network configuration or NetworkManager + NetworkManager-config-server.<br>
<br>
<br>
Corosync 3.x with knet will work for sure, but be prepared for quite a long compile path, because you first have to compile knet and then corosync. What may help you a bit is that we have a ubuntu 18.04 in our jenkins, so it should be possible corosync build log <a href="https://ci.kronosnet.org/view/corosync/job/corosync-build-all-voting/lastBuild/corosync-build-all-voting=ubuntu-18-04-lts-x86-64/consoleText" rel="noreferrer" target="_blank">https://ci.kronosnet.org/view/<wbr>corosync/job/corosync-build-al<wbr>l-voting/lastBuild/corosync-bu<wbr>ild-all-voting=ubuntu-18-04-lt<wbr>s-x86-64/consoleText</a>, knet build log <a href="https://ci.kronosnet.org/view/knet/job/knet-build-all-voting/lastBuild/knet-build-all-voting=ubuntu-18-04-lts-x86-64/consoleText" rel="noreferrer" target="_blank">https://ci.kronosnet.org/view/<wbr>knet/job/knet-build-all-voting<wbr>/lastBuild/knet-build-all-<wbr>voting=ubuntu-18-04-lts-x86-<wbr>64/consoleText</a>).<br>
<br>
Also please consult <a href="http://people.redhat.com/ccaulfie/docs/KnetCorosync.pdf" rel="noreferrer" target="_blank">http://people.redhat.com/ccaul<wbr>fie/docs/KnetCorosync.pdf</a> about changes in corosync configuration.<br>
<br>
Regards,<br>
Honza<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
<br>
Thank you for your quick response!<br>
<br>
2018-08-23 8:40 GMT+02:00 Jan Friesse <<a href="mailto:jfriesse@redhat.com" target="_blank">jfriesse@redhat.com</a>>:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
David,<br>
<br>
Hello,<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Im getting crazy about this problem, that I expect to resolve here, with<br>
your help guys:<br>
<br>
I have 2 nodes with Corosync redundant ring feature.<br>
<br>
Each node has 2 similarly connected/configured NIC's. Both nodes are<br>
connected each other by two crossover cables.<br>
<br>
</blockquote>
<br>
I believe this is root of the problem. Are you using NetworkManager? If<br>
so, have you installed NetworkManager-config-server? If not, please install<br>
it and test again.<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I configured both nodes with rrp mode passive. Everything is working well<br>
at this point, but when I shutdown 1 node to test failover, and this<br>
node > returns to be online, corosync is marking the interface as FAULTY<br>
and rrp<br>
<br>
</blockquote>
<br>
I believe it's because with crossover cables configuration when other<br>
side is shutdown, NetworkManager detects it and does ifdown of the<br>
interface. And corosync is unable to handle ifdown properly. Ifdown is bad<br>
with single ring, but it's just killer with RRP (127.0.0.1 poisons every<br>
node in the cluster).<br>
<br>
fails to recover the initial state:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
1. Initial scenario:<br>
<br>
# corosync-cfgtool -s<br>
Printing ring status.<br>
Local node ID 1<br>
RING ID 0<br>
id = 192.168.0.1<br>
status = ring 0 active with no faults<br>
RING ID 1<br>
id = 192.168.1.1<br>
status = ring 1 active with no faults<br>
<br>
<br>
2. When I shutdown the node 2, all continues with no faults. Sometimes<br>
the<br>
ring ID's are bonding with 127.0.0.1 and then bond back to their<br>
respective<br>
heartbeat IP.<br>
<br>
</blockquote>
<br>
Again, result of ifdown.<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
3. When node 2 is back online:<br>
<br>
# corosync-cfgtool -s<br>
Printing ring status.<br>
Local node ID 1<br>
RING ID 0<br>
id = 192.168.0.1<br>
status = ring 0 active with no faults<br>
RING ID 1<br>
id = 192.168.1.1<br>
status = Marking ringid 1 interface 192.168.1.1 FAULTY<br>
<br>
<br>
# service corosync status<br>
● corosync.service - Corosync Cluster Engine<br>
Loaded: loaded (/lib/systemd/system/corosync.<wbr>service; enabled;<br>
vendor<br>
preset: enabled)<br>
Active: active (running) since Wed 2018-08-22 14:44:09 CEST; 1min<br>
38s ago<br>
Docs: man:corosync<br>
man:corosync.conf<br>
man:corosync_overview<br>
Main PID: 1439 (corosync)<br>
Tasks: 2 (limit: 4915)<br>
CGroup: /system.slice/corosync.service<br>
└─1439 /usr/sbin/corosync -f<br>
<br>
<br>
Aug 22 14:44:11 node1 corosync[1439]: Aug 22 14:44:11 notice [TOTEM ]<br>
The<br>
network interface [192.168.0.1] is now up.<br>
Aug 22 14:44:11 node1 corosync[1439]: [TOTEM ] The network interface<br>
[192.168.0.1] is now up.<br>
Aug 22 14:44:11 node1 corosync[1439]: Aug 22 14:44:11 notice [TOTEM ]<br>
The<br>
network interface [192.168.1.1] is now up.<br>
Aug 22 14:44:11 node1 corosync[1439]: [TOTEM ] The network interface<br>
[192.168.1.1] is now up.<br>
Aug 22 14:44:26 node1 corosync[1439]: Aug 22 14:44:26 notice [TOTEM ] A<br>
new membership (<a href="http://192.168.0.1:601760" rel="noreferrer" target="_blank">192.168.0.1:601760</a>) was formed. Members<br>
Aug 22 14:44:26 node1 corosync[1439]: [TOTEM ] A new membership (<br>
<a href="http://192.168.0.1:601760" rel="noreferrer" target="_blank">192.168.0.1:601760</a>) was formed. Members<br>
Aug 22 14:44:32 node1 corosync[1439]: Aug 22 14:44:32 notice [TOTEM ] A<br>
new membership (<a href="http://192.168.0.1:601764" rel="noreferrer" target="_blank">192.168.0.1:601764</a>) was formed. Members joined: 2<br>
Aug 22 14:44:32 node1 corosync[1439]: [TOTEM ] A new membership (<br>
<a href="http://192.168.0.1:601764" rel="noreferrer" target="_blank">192.168.0.1:601764</a>) was formed. Members joined: 2<br>
Aug 22 14:44:34 node1 corosync[1439]: Aug 22 14:44:34 error [TOTEM ]<br>
Marking ringid 1 interface 192.168.1.1 FAULTY<br>
Aug 22 14:44:34 node1 corosync[1439]: [TOTEM ] Marking ringid 1<br>
interface<br>
192.168.1.1 FAULTY<br>
<br>
<br>
If I execute corosync-cfgtool, clears the faulty error but after some<br>
seconds return to be FAULTY.<br>
The only thing that it resolves the problem is to restart de service with<br>
service corosync restart.<br>
<br>
Here you have some of my configuration settings on node 1 (I probed<br>
already<br>
to change rrp_mode):<br>
<br>
*- corosync.conf*<br>
<br>
<br>
totem {<br>
version: 2<br>
cluster_name: node<br>
token: 5000<br>
token_retransmits_before_loss_<wbr>const: 10<br>
secauth: off<br>
threads: 0<br>
rrp_mode: passive<br>
nodeid: 1<br>
interface {<br>
ringnumber: 0<br>
bindnetaddr: 192.168.0.0<br>
#mcastaddr: 226.94.1.1<br>
mcastport: 5405<br>
broadcast: yes<br>
}<br>
interface {<br>
ringnumber: 1<br>
bindnetaddr: 192.168.1.0<br>
#mcastaddr: 226.94.1.2<br>
mcastport: 5407<br>
broadcast: yes<br>
}<br>
}<br>
<br>
logging {<br>
fileline: off<br>
to_stderr: yes<br>
to_syslog: yes<br>
to_logfile: yes<br>
logfile: /var/log/corosync/corosync.log<br>
debug: off<br>
timestamp: on<br>
logger_subsys {<br>
subsys: AMF<br>
debug: off<br>
}<br>
}<br>
<br>
amf {<br>
mode: disabled<br>
}<br>
<br>
quorum {<br>
provider: corosync_votequorum<br>
expected_votes: 2<br>
}<br>
<br>
nodelist {<br>
node {<br>
nodeid: 1<br>
ring0_addr: 192.168.0.1<br>
ring1_addr: 192.168.1.1<br>
}<br>
<br>
node {<br>
nodeid: 2<br>
ring0_addr: 192.168.0.2<br>
ring1_addr: 192.168.1.2<br>
}<br>
}<br>
<br>
aisexec {<br>
user: root<br>
group: root<br>
}<br>
<br>
service {<br>
name: pacemaker<br>
ver: 1<br>
}<br>
<br>
<br>
<br>
*- /etc/hosts*<br>
<br>
<br>
127.0.0.1 localhost<br>
10.4.172.5 <a href="http://node1.upc.edu" rel="noreferrer" target="_blank">node1.upc.edu</a> node1<br>
10.4.172.6 <a href="http://node2.upc.edu" rel="noreferrer" target="_blank">node2.upc.edu</a> node2<br>
<br>
<br>
</blockquote>
So machines have 3 NICs? 2 for corosync/cluster traffic and one for<br>
regular traffic/services/outside world?<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thank you for you help in advance!<br>
<br>
</blockquote>
<br>
To conclude:<br>
- If you are using NetworkManager, try to install<br>
NetworkManager-config-server, it will probably help<br>
- If you are brave enough, try corosync 3.x (current Alpha4 is pretty<br>
stable - actually some other projects gain this stability with SP1 :) )<br>
that has no RRP but uses knet for support redundant links (up-to 8 links<br>
can be configured) and doesn't have problems with ifdown.<br>
<br>
Honza<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
______________________________<wbr>_________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank">Users@clusterlabs.org</a><br>
<a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.clusterlabs.org/<wbr>mailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">http://www.clusterlabs.org/doc<wbr>/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
<br>
<br>
</blockquote>
<br>
</blockquote>
<br>
<br>
--<br></div></div>
*David Tolosa Martínez*<span class=""><br>
Customer Support & Infrastructure<br>
UPCnet - Edifici Vèrtex<br>
Plaça d'Eusebi Güell, 6, 08034 Barcelona<br>
Tel: 934054555<br>
<br></span>
<<a href="https://www.upcnet.es" rel="noreferrer" target="_blank">https://www.upcnet.es</a>><br>
<br>
</blockquote>
<br>
<br>
<br>
</blockquote>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr" style="font-size:12.8px"><div><div><b><font color="#666666">David Tolosa Martínez</font></b><br></div><span style="color:rgb(136,136,136);font-size:12.8px">Customer Support & Infrastructure</span><br></div><font color="#888888">UPCnet - Edifici Vèrtex</font></div><div dir="ltr" style="font-size:12.8px"><span style="color:rgb(136,136,136);font-size:12.8px">Plaça d'Eusebi Güell, 6, 08034 Barcelona</span><font color="#888888"><br>Tel: 934054555</font><br><br></div><div dir="ltr"><a href="https://www.upcnet.es" target="_blank"><img src="https://www.upcnet.es/ca/img/upcnet.png"></a><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
<br>
<br><br><p dir="ltr" style="line-height:1.3090909090909089;margin-top:0pt;margin-bottom:0pt;border-top:solid #eeeeee 0.75pt;background-color:#ffffff;padding:11.25pt 0pt 0pt 0pt"><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap">INFORMACIÓ BÀSICA SOBRE PROTECCIÓ DE DADES:</span></p><p dir="ltr" style="line-height:1.3090909090909089;margin-top:0pt;margin-bottom:0pt;background-color:#ffffff"> </p><p dir="ltr" style="line-height:1.3090909090909089;margin-top:0pt;margin-bottom:0pt;background-color:#ffffff"><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap">Responsable:</span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap"> UPCNET, Serveis d'Accés a Internet de la Universitat Politècnica de Catalunya, SLU | </span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap">Finalitat:</span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap"> gestionar els contactes i les relacions professionals i comercials amb els nostres clients i proveïdors | </span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap">Base legal:</span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap"> consentiment, interès legítim i/o relació contractual | </span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap">Destinataris:</span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap"> no seran comunicades a tercers excepte per obligació legal | </span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap">Drets:</span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap"> pots exercir els teus drets d’accés, rectificació i supressió, així com els altres drets reconeguts a la normativa vigent, enviant-nos un missatge a </span><span style="font-size:8.5pt;font-family:Arial;color:#207ebe;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap"><a href="mailto:privacy@upcnet.es" target="_blank">privacy@upcnet.es</a></span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap"> | </span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap">Més informació</span><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap">: consulta la nostra </span><a href="https://www.upcnet.es/politica-de-privacitat" style="text-decoration:none" target="_blank"><span style="font-size:8.5pt;font-family:Arial;color:#207ebe;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline;white-space:pre;white-space:pre-wrap">política completa de protecció de dades</span></a><span style="font-size:8.5pt;font-family:Arial;color:#999999;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap">.</span></p><p dir="ltr" style="line-height:1.3090909090909089;margin-top:0pt;margin-bottom:0pt;background-color:#ffffff"> </p><p dir="ltr" style="line-height:1.3090909090909089;margin-top:0pt;margin-bottom:0pt;background-color:#ffffff"><a href="https://www.upcnet.es/ca/avis-de-confidencialitat" style="text-decoration:none" target="_blank"><span style="font-size:8.5pt;font-family:Arial;color:#207ebe;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline;white-space:pre;white-space:pre-wrap">AVÍS DE CONFIDENCIALITAT</span></a></p>