<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:st="" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-GB link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal>Greetings everyone,<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I am currently experiencing some troubles getting two
cluster nodes in a corosync + pacemaker set up to talk with / find each other
and set up properly.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I have read various howtos out in the net as well as based
on the set up I’ve done last year too, but to no avail.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Basically I have two VMs in a VLAN, which has all necessary
ports open inside the VLAN, and both are supposed to be clustered for an NFS
system (which will involve DRBD too).<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>So far the problem appears to be on the corosync / pacemaker
level as in the nodes can see each other (based on what the logs state), yet
running ‘crm configure’ and setting various properties, the error
always is that the remote node cannot be found. I am not able to get past this
problem, and am running out of ideas.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Short and FQDN hostname are all set up too.<o:p></o:p></p>
<p class=MsoNormal>‘crm configure’ is run as root which is also part
of the ‘haclient’ group. <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>/etc/hosts.allow and hosts.deny do not have any entries
preventing connections<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I’d be grateful for any ideas / advise / help etc. <span
style='font-family:Wingdings'>J</span><o:p></o:p></p>
<p class=MsoNormal>Please let me know if any further logs or missing
configuration print outs are needed. Thanks.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>The package versions are:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>rpm -qa | egrep
"corosync|pacemaker|cluster|resource"<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>corosynclib-1.4.1-4.el6.x86_64<o:p></o:p></p>
<p class=MsoNormal>clusterlib-3.0.12.1-23.el6.x86_64<o:p></o:p></p>
<p class=MsoNormal>pacemaker-cluster-libs-1.1.6-3.el6.x86_64<o:p></o:p></p>
<p class=MsoNormal>resource-agents-3.9.2-7.el6.x86_64<o:p></o:p></p>
<p class=MsoNormal>cluster-glue-libs-1.0.5-2.el6.x86_64<o:p></o:p></p>
<p class=MsoNormal>pacemaker-libs-1.1.6-3.el6.x86_64<o:p></o:p></p>
<p class=MsoNormal>corosync-1.4.1-4.el6.x86_64<o:p></o:p></p>
<p class=MsoNormal>pacemaker-cli-1.1.6-3.el6.x86_64<o:p></o:p></p>
<p class=MsoNormal>cluster-glue-1.0.5-2.el6.x86_64<o:p></o:p></p>
<p class=MsoNormal>pacemaker-1.1.6-3.el6.x86_64<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Below are the various configuration files and log messages:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>### /etc/corosync/corosync.conf ###<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>compatibility: whitetank<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>totem {<o:p></o:p></p>
<p class=MsoNormal> version: 2<o:p></o:p></p>
<p class=MsoNormal> secauth: off<o:p></o:p></p>
<p class=MsoNormal> threads: 0<o:p></o:p></p>
<p class=MsoNormal> join:
1000<o:p></o:p></p>
<p class=MsoNormal> consensus: 7500<o:p></o:p></p>
<p class=MsoNormal> max_messages: 20<o:p></o:p></p>
<p class=MsoNormal> interface {<o:p></o:p></p>
<p class=MsoNormal>
ringnumber: 0<o:p></o:p></p>
<p class=MsoNormal style='text-indent:36.0pt'>bindnetaddr: 10.26.29.0 # colleague
set these to .238 and .239 respectively on the other node, which are their eth0
NICs to test if it makes any difference<o:p></o:p></p>
<p class=MsoNormal style='text-indent:36.0pt'>mcastaddr: 226.94.1.1<o:p></o:p></p>
<p class=MsoNormal style='text-indent:36.0pt'>mcastport: 5405<o:p></o:p></p>
<p class=MsoNormal>
#ttl: 1<o:p></o:p></p>
<p class=MsoNormal> }<o:p></o:p></p>
<p class=MsoNormal>}<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>logging {<o:p></o:p></p>
<p class=MsoNormal> fileline: off<o:p></o:p></p>
<p class=MsoNormal> to_stderr: off<o:p></o:p></p>
<p class=MsoNormal> to_logfile: yes<o:p></o:p></p>
<p class=MsoNormal> to_syslog: yes<o:p></o:p></p>
<p class=MsoNormal> logfile:
/var/log/cluster/corosync.log<o:p></o:p></p>
<p class=MsoNormal> debug: on<o:p></o:p></p>
<p class=MsoNormal> timestamp: on<o:p></o:p></p>
<p class=MsoNormal> logger_subsys {<o:p></o:p></p>
<p class=MsoNormal>
subsys: AMF<o:p></o:p></p>
<p class=MsoNormal>
debug: off<o:p></o:p></p>
<p class=MsoNormal> }<o:p></o:p></p>
<p class=MsoNormal>}<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>amf {<o:p></o:p></p>
<p class=MsoNormal> mode: disabled<o:p></o:p></p>
<p class=MsoNormal>}<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>### /etc/corosync/service.d/pcmk ###<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>service<o:p></o:p></p>
<p class=MsoNormal>{<o:p></o:p></p>
<p class=MsoNormal> name: pacemaker<o:p></o:p></p>
<p class=MsoNormal> ver: 1<o:p></o:p></p>
<p class=MsoNormal> use_mgmtd: no<o:p></o:p></p>
<p class=MsoNormal> use_logd: yes<o:p></o:p></p>
<p class=MsoNormal>}<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>### /etc/sysconfig/pacemaker (all default) ###<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal># Variables for running child daemons under valgrind and/or
checking for memory problems<o:p></o:p></p>
<p class=MsoNormal>#export G_SLICE=always-malloc<o:p></o:p></p>
<p class=MsoNormal>#export MALLOC_PERTURB_=221 # or 0<o:p></o:p></p>
<p class=MsoNormal>#export MALLOC_CHECK_=3 # or 0,1,2<o:p></o:p></p>
<p class=MsoNormal>#export HA_valgrind_enabled=yes<o:p></o:p></p>
<p class=MsoNormal>#export HA_valgrind_enabled=cib,crmd<o:p></o:p></p>
<p class=MsoNormal>#export HA_callgrind_enabled=yes<o:p></o:p></p>
<p class=MsoNormal>#export HA_callgrind_enabled=cib,crmd<o:p></o:p></p>
<p class=MsoNormal>#export VALGRIND_OPTS="--leak-check=full
--trace-children=no --num-callers=25
--log-file=/tmp/pacemaker-%p.valgrind"<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal># Variables that control logging<o:p></o:p></p>
<p class=MsoNormal>#export PCMK_trace_functions=<o:p></o:p></p>
<p class=MsoNormal>#export PCMK_trace_formats=<o:p></o:p></p>
<p class=MsoNormal>#export PCMK_trace_files=<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>### crm error ###<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>crm(live)configure# property
stonith-enabled="false"<o:p></o:p></p>
<p class=MsoNormal>crm(live)configure# commit<o:p></o:p></p>
<p class=MsoNormal>Call cib_replace failed (-41): Remote node did not respond<o:p></o:p></p>
<p class=MsoNormal><null><o:p></o:p></p>
<p class=MsoNormal>ERROR: could not replace cib<o:p></o:p></p>
<p class=MsoNormal>INFO: offending xml: <configuration><o:p></o:p></p>
<p class=MsoNormal>
<crm_config><o:p></o:p></p>
<p class=MsoNormal>
<cluster_property_set id="cib-bootstrap-options"><o:p></o:p></p>
<p class=MsoNormal>
<nvpair id="cib-bootstrap-options-stonith-enabled"
name="stonith-enabled" value="false"/><o:p></o:p></p>
<p class=MsoNormal>
</cluster_property_set><o:p></o:p></p>
<p class=MsoNormal>
</crm_config><o:p></o:p></p>
<p class=MsoNormal> <nodes/><o:p></o:p></p>
<p class=MsoNormal>
<resources/><o:p></o:p></p>
<p class=MsoNormal>
<constraints/><o:p></o:p></p>
<p class=MsoNormal></configuration><o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>### netstat node 1 ###<o:p></o:p></p>
<p class=MsoNormal>netstat -tulpen<o:p></o:p></p>
<p class=MsoNormal>udp
0 0
10.26.29.238:5404
0.0.0.0:*
0
141073
9501/corosync <o:p></o:p></p>
<p class=MsoNormal>udp
0 0
10.26.29.238:5405
0.0.0.0:*
0 141074
9501/corosync <o:p></o:p></p>
<p class=MsoNormal>udp
0 0
226.94.1.1:5405
0.0.0.0:*
0
141072 9501/corosync<o:p></o:p></p>
<p class=MsoNormal>(and same for node 2 with its respective ip 10.26.29.239)<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>netstat -nlpa | grep corosync<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>udp
0 0 10.26.29.238:5404
0.0.0.0:*
9501/corosync <o:p></o:p></p>
<p class=MsoNormal>udp
0 0
10.26.29.238:5405
0.0.0.0:*
9501/corosync <o:p></o:p></p>
<p class=MsoNormal>udp
0 0
226.94.1.1:5405
0.0.0.0:* 9501/corosync
<o:p></o:p></p>
<p class=MsoNormal>unix 2 [ ACC
] STREAM
LISTENING 141067
9501/corosync @corosync.ipc<o:p></o:p></p>
<p class=MsoNormal>unix 3 [
]
STREAM CONNECTED 141236
9501/corosync @corosync.ipc<o:p></o:p></p>
<p class=MsoNormal>unix 3 [
] STREAM
CONNECTED 141225
9501/corosync @corosync.ipc<o:p></o:p></p>
<p class=MsoNormal>unix 3 [
]
STREAM CONNECTED 141200
9501/corosync @corosync.ipc<o:p></o:p></p>
<p class=MsoNormal>unix 3 [
]
STREAM CONNECTED 141161
9501/corosync @corosync.ipc<o:p></o:p></p>
<p class=MsoNormal>unix 3 [
]
STREAM CONNECTED 141152
9501/corosync @corosync.ipc<o:p></o:p></p>
<p class=MsoNormal>unix 3 [
]
STREAM CONNECTED 141132
9501/corosync @corosync.ipc<o:p></o:p></p>
<p class=MsoNormal>unix 3 [
]
STREAM CONNECTED 141124
9501/corosync @corosync.ipc<o:p></o:p></p>
<p class=MsoNormal>unix 2 [
]
DGRAM
141064 9501/corosync <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>### /var/log/messages ### <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[MAIN ] Corosync Cluster Engine ('1.4.1'): started and ready to provide
service.<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[MAIN ] Corosync built-in features: nss dbus rdma snmp<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[MAIN ] Successfully read main configuration file
'/etc/corosync/corosync.conf'.<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[TOTEM ] Initializing transport (UDP/IP Multicast).<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC
(mode 0).<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[TOTEM ] The network interface [10.26.29.238] is now up.<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[SERV ] Service engine loaded: corosync extended virtual synchrony
service<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[SERV ] Service engine loaded: corosync configuration service<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[SERV ] Service engine loaded: corosync cluster closed process group
service v1.01<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[SERV ] Service engine loaded: corosync cluster config database access
v1.01<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[SERV ] Service engine loaded: corosync profile loading service<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[SERV ] Service engine loaded: corosync cluster quorum service v0.1<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[MAIN ] Compatibility mode set to whitetank. Using V1 and V2 of the
synchronization engine.<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[TOTEM ] A processor joined or left the membership and a new membership was
formed.<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[CPG ] chosen downlist: sender r(0) ip(10.26.29.238) ;
members(old:0 left:0)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[MAIN ] Completed service synchronization, ready to provide service.<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[TOTEM ] A processor joined or left the membership and a new membership was
formed.<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[CPG ] chosen downlist: sender r(0) ip(10.26.29.238) ;
members(old:1 left:0)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:23 wkse13p1xynfs01 corosync[9501]:
[MAIN ] Completed service synchronization, ready to provide service.<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
Invoked: pacemakerd <o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/root<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
config_find_next: No additional configuration supplied for: service<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
config_find_next: No additional configuration supplied for: quorum<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
get_config_opt: No default for option: provider<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
get_cluster_type: Detected an active 'corosync' cluster<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
read_config: Reading configure for stack: corosync<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
config_find_next: Processing additional logging options...<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
get_config_opt: Found 'on' for option: debug<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
get_config_opt: Found 'yes' for option: to_logfile<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
get_config_opt: Found '/var/log/cluster/corosync.log' for option: logfile<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
get_config_opt: Found 'yes' for option: to_syslog<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9514]: info:
get_config_opt: Defaulting to 'daemon' for option: syslog_facility<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/root<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
main: Starting Pacemaker 1.1.6-3.el6 (Build:
a02c0f19a00c1eb2527ad38f146ebc0834814558): generated-manpages
agent-manpages ascii-docs publican-docs ncurses trace-logging cman
corosync-quorum corosync<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
main: Maximum core file size is: 18446744073709551615<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: 0x1f41220 Node 3994884618 now known as wkse13p1xynfs01
(was: (null))<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs01 now has process list:
00000000000000000000000000000002 (was 00000000000000000000000000000000)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
G_main_add_SignalHandler: Added signal handler for signal 17<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
start_child: Forked child 9521 for process stonith-ng<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs01 now has process list:
00000000000000000000000000100002 (was 00000000000000000000000000000002)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
start_child: Forked child 9522 for process cib<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs01 now has process list:
00000000000000000000000000100102 (was 00000000000000000000000000100002)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info: start_child:
Forked child 9523 for process lrmd<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs01 now has process list:
00000000000000000000000000100112 (was 00000000000000000000000000100102)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
start_child: Forked child 9524 for process attrd<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs01 now has process list:
00000000000000000000000000101112 (was 00000000000000000000000000100112)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
start_child: Forked child 9525 for process pengine<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs01 now has process list:
00000000000000000000000000111112 (was 00000000000000000000000000101112)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
start_child: Forked child 9526 for process crmd<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs01 now has process list:
00000000000000000000000000111312 (was 00000000000000000000000000111112)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
main: Starting mainloop<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: 0x1f43510 Node 4011661834 now known as wkse13p1xynfs02
(was: (null))<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs02 now has process list:
00000000000000000000000000000002 (was 00000000000000000000000000000000)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs02 now has process list:
00000000000000000000000000100002 (was 00000000000000000000000000000002)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs02 now has process list:
00000000000000000000000000100102 (was 00000000000000000000000000100002)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs02 now has process list:
00000000000000000000000000100112 (was 00000000000000000000000000100102)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs02 now has process list:
00000000000000000000000000101112 (was 00000000000000000000000000100112)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 lrmd: [9523]: info:
G_main_add_SignalHandler: Added signal handler for signal 15<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs02 now has process list:
00000000000000000000000000111112 (was 00000000000000000000000000101112)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 stonith-ng: [9521]: info:
Invoked: /usr/lib64/heartbeat/stonithd <o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pacemakerd: [9517]: info:
update_node_processes: Node wkse13p1xynfs02 now has process list:
00000000000000000000000000111312 (was 00000000000000000000000000111112)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 stonith-ng: [9521]: info:
crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/root<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 stonith-ng: [9521]: info:
G_main_add_SignalHandler: Added signal handler for signal 17<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info:
crm_log_init_worker: Changed active directory to
/var/lib/heartbeat/cores/hacluster<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info:
G_main_add_TriggerHandler: Added signal manual handler<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info:
G_main_add_SignalHandler: Added signal handler for signal 17<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 lrmd: [9523]: info:
G_main_add_SignalHandler: Added signal handler for signal 17<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 lrmd: [9523]: info: enabling
coredumps<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 lrmd: [9523]: info:
G_main_add_SignalHandler: Added signal handler for signal 10<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 lrmd: [9523]: info:
G_main_add_SignalHandler: Added signal handler for signal 12<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 lrmd: [9523]: info: Started.<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 stonith-ng: [9521]: info:
get_cluster_type: Cluster type is: 'corosync'<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 stonith-ng: [9521]: notice:
crm_cluster_connect: Connecting to cluster infrastructure: corosync<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: info:
Invoked: /usr/lib64/heartbeat/attrd <o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info:
retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml
(digest: /var/lib/heartbeat/crm/cib.xml.sig)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info:
validate_with_relaxng: Creating RNG parser context<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: info:
crm_log_init_worker: Changed active directory to
/var/lib/heartbeat/cores/hacluster<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: info: main:
Starting up<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: info:
get_cluster_type: Cluster type is: 'corosync'<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: notice:
crm_cluster_connect: Connecting to cluster infrastructure: corosync<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 crmd: [9526]: info: Invoked:
/usr/lib64/heartbeat/crmd <o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 crmd: [9526]: info:
crm_log_init_worker: Changed active directory to
/var/lib/heartbeat/cores/hacluster<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pengine: [9525]: info:
Invoked: /usr/lib64/heartbeat/pengine <o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pengine: [9525]: info:
crm_log_init_worker: Changed active directory to
/var/lib/heartbeat/cores/hacluster<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 crmd: [9526]: info: main:
CRM Hg Version: a02c0f19a00c1eb2527ad38f146ebc0834814558<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 crmd: [9526]: info:
crmd_init: Starting crmd<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 crmd: [9526]: info:
G_main_add_SignalHandler: Added signal handler for signal 17<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 pengine: [9525]: info: main:
Starting pengine<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 stonith-ng: [9521]: info:
init_ais_connection_once: Connection to 'corosync': established<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 stonith-ng: [9521]: info:
crm_new_peer: Node wkse13p1xynfs01 now has id: 3994884618<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 stonith-ng: [9521]: info:
crm_new_peer: Node 3994884618 is now known as wkse13p1xynfs01<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 stonith-ng: [9521]: info:
main: Starting stonith-ng mainloop<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 stonith-ng: [9521]: info:
crm_update_peer: Node wkse13p1xynfs01: id=3994884618 state=unknown addr=(null)
votes=0 born=0 seen=0 proc=00000000000000000000000000111312 (new)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 stonith-ng: [9521]: info: crm_new_peer:
Node 0 is now known as wkse13p1xynfs02<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: info:
init_ais_connection_once: Connection to 'corosync': established<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: info:
crm_new_peer: Node wkse13p1xynfs01 now has id: 3994884618<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: info:
crm_new_peer: Node 3994884618 is now known as wkse13p1xynfs01<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: info: main:
Cluster connection active<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: info: main:
Accepting attribute updates<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: notice: main:
Starting mainloop...<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: info:
crm_update_peer: Node wkse13p1xynfs01: id=3994884618 state=unknown addr=(null)
votes=0 born=0 seen=0 proc=00000000000000000000000000111312 (new)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 attrd: [9524]: info:
crm_new_peer: Node 0 is now known as wkse13p1xynfs02<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info: startCib:
CIB Initialization completed successfully<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info:
get_cluster_type: Cluster type is: 'corosync'<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: notice:
crm_cluster_connect: Connecting to cluster infrastructure: corosync<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info:
init_ais_connection_once: Connection to 'corosync': established<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info:
crm_new_peer: Node wkse13p1xynfs01 now has id: 3994884618<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info:
crm_new_peer: Node 3994884618 is now known as wkse13p1xynfs01<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info: cib_init:
Starting cib mainloop<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info:
crm_update_peer: Node wkse13p1xynfs01: id=3994884618 state=unknown addr=(null)
votes=0 born=0 seen=0 proc=00000000000000000000000000111312 (new)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info:
crm_new_peer: Node 0 is now known as wkse13p1xynfs02<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:29 wkse13p1xynfs01 cib: [9522]: info: Managed
write_cib_contents process 9530 exited with return code 0.<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
do_cib_control: CIB connection established<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
get_cluster_type: Cluster type is: 'corosync'<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: notice:
crm_cluster_connect: Connecting to cluster infrastructure: corosync<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
init_ais_connection_once: Connection to 'corosync': established<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
crm_new_peer: Node wkse13p1xynfs01 now has id: 3994884618<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
crm_new_peer: Node 3994884618 is now known as wkse13p1xynfs01<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
ais_status_callback: status: wkse13p1xynfs01 is now unknown<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
init_quorum_connection: Configuring Pacemaker to obtain quorum from Corosync<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: notice: init_quorum_connection:
Quorum acquired<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
do_ha_control: Connected to the cluster<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
do_started: Delaying start, no membership data (0000000000100000)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
crmd_init: Starting crmd's mainloop<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
config_query_callback: Shutdown escalation occurs after: 1200000ms<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info: config_query_callback:
Checking for expired actions every 900000ms<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: notice:
crmd_peer_update: Status update: Client wkse13p1xynfs01/crmd now has status
[online] (DC=<null>)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
crm_update_peer: Node wkse13p1xynfs01: id=3994884618 state=unknown addr=(null)
votes=0 born=0 seen=0 proc=00000000000000000000000000111312 (new)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
crm_new_peer: Node 0 is now known as wkse13p1xynfs02<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
ais_status_callback: status: wkse13p1xynfs02 is now unknown<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
pcmk_quorum_notification: Membership 0: quorum retained (0)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:30 wkse13p1xynfs01 crmd: [9526]: info:
do_started: Delaying start, no membership data (0000000000100000)<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:34 wkse13p1xynfs01 attrd: [9524]: info:
cib_connect: Connected to the CIB after 1 signon attempts<o:p></o:p></p>
<p class=MsoNormal>Mar 14 18:01:34 wkse13p1xynfs01 attrd: [9524]: info:
cib_connect: Sending full refresh<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Cheers,<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Achim<o:p></o:p></p>
</div>
</body>
</html>