<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body text="#000000" bgcolor="#ffffff">
<div class="moz-text-flowed"
style="font-family: -moz-fixed; font-size: 12px;" lang="x-western">Hello,
<br>
<br>
I have a working config for a drbd/nfs active/passive cluster.
<br>
<br>
I added a webserver to the cluster. This webserver is hosted in a
Fedora 13 VM.
<br>
Nodes communicate in a vlan (physical and virtual nodes communicate
directly through the vlan).
<br>
I configured a first webserver node, and everything went fine. It goes
online, start httpd (and pureftpd). A i said, this httpd node is a kvm
virtual machine.
<br>
DRBD/nfs active/passive are both physical nodes (under gentoo)
<br>
Every node are running corosync 1.2.3, fedora use pacemaker 1.1.1 and
gentoo pacemaker 1.0.8
<br>
<br>
Here is the cib :
<br>
node filer1.connecting-nature.com \
<br>
attributes standby="off"
<br>
node filer2.connecting-nature.com \
<br>
attributes standby="off"
<br>
node www01.connecting-nature.com \
<br>
attributes standby="off"
<br>
node www02.connecting-nature.com \
<br>
attributes standby="off"
<br>
primitive drbd_nfs ocf:linbit:drbd \
<br>
params drbd_resource="r0" \
<br>
op monitor interval="59s" role="Master" timeout="60s" \
<br>
op monitor interval="60s" role="Slave" timeout="60s"
<br>
primitive fs_nfs ocf:heartbeat:Filesystem \
<br>
op monitor interval="120s" timeout="60s" \
<br>
params device="/dev/drbd0" directory="/data" fstype="ext4"
<br>
primitive httpd_server ocf:heartbeat:apache \
<br>
op monitor interval="40s" timeout="20s" \
<br>
params configfile="/etc/httpd/conf/httpd.conf"
<br>
primitive ip_failover heartbeat:OVHfailover.py \
<br>
op monitor interval="120s" timeout="60s" \
<br>
params 1="xxxxxx-ovh" 2="xxxxxx" 3="nsxxxxx.ovh.net"
4="ip.fail.ov.er" \
<br>
meta target-role="Started"
<br>
primitive ip_nfs ocf:heartbeat:IPaddr2 \
<br>
op monitor interval="60s" timeout="20s" \
<br>
params ip="192.168.0.20" cidr_netmask="24" nic="vlan2019"
<br>
primitive nfs_server lsb:nfs \
<br>
op monitor interval="120s" timeout="60s"
<br>
primitive pureftpd_server ocf:heartbeat:Pure-FTPd \
<br>
op monitor interval="30s" timeout="10s" \
<br>
params script="/usr/sbin/pure-config.pl" daemon_type="ldap"
<br>
group group_nfs ip_nfs fs_nfs nfs_server ip_failover \
<br>
meta target-role="Started"
<br>
ms ms_drbd_nfs drbd_nfs \
<br>
meta master-max="1" master-node-max="1" clone-max="2"
clone-node-max="1" notify="true" tar
<br>
ms ms_pureftpd_server pureftpd_server \
<br>
meta master-max="1" master-node-max="1" clone-max="2"
clone-node-max="1" notify="true" tar
<br>
clone httpd-clone httpd_server
<br>
location drbd-filer1 ms_drbd_nfs \
<br>
rule $id="drbd-filer1-rule" 90: #uname eq
filer1.connecting-nature.com
<br>
location drbd-filer2 ms_drbd_nfs \
<br>
rule $id="drbd-filer2-rule" 90: #uname eq
filer2.connecting-nature.com
<br>
location httpd-www01 httpd-clone \
<br>
rule $id="httpd-www01-rule" 90: #uname eq
www01.connecting-nature.com
<br>
location httpd-www02 httpd-clone \
<br>
rule $id="httpd-www02-rule" 90: #uname eq
www02.connecting-nature.com
<br>
location nfs-filer1 group_nfs \
<br>
rule $id="nfs-filer1-rule" 90: #uname eq
filer1.connecting-nature.com
<br>
location nfs-filer2 group_nfs \
<br>
rule $id="nfs-filer2-rule" 90: #uname eq
filer2.connecting-nature.com
<br>
location pureftpd_www01 ms_pureftpd_server \
<br>
rule $id="pureftpd-www01-rule" 90: #uname eq
www01.connecting-nature.com
<br>
location pureftpd_www02 ms_pureftpd_server \
<br>
rule $id="pureftpd-www02-rule" 90: #uname eq
www02.connecting-nature.com
<br>
colocation nfs_on_drbd inf: group_nfs ms_drbd_nfs:Master
<br>
order nfs_after_drbd inf: ms_drbd_nfs:promote group_nfs:start
<br>
property $id="cib-bootstrap-options" \
<br>
symmetric-cluster="false" \
<br>
no-quorum-policy="ignore" \
<br>
default-resource-stickiness="100" \
<br>
default-resource-failure-stickiness="10" \
<br>
stonith-enabled="false" \
<br>
stonith-action="reboot" \
<br>
stop-orphan-resources="true" \
<br>
stop-orphan-actions="true" \
<br>
remove-after-stop="false" \
<br>
short-resource-names="true" \
<br>
transition-idle-timeout="3min" \
<br>
default-action-timeout="30s" \
<br>
is-managed-default="true" \
<br>
startup-fencing="true" \
<br>
cluster-delay="60s" \
<br>
expected-nodes="1" \
<br>
expected-quorum-votes="4" \
<br>
dc-version="1.0.8-9881a7350d6182bae9e8e557cf20a3cc5dac3ee7" \
<br>
cluster-infrastructure="openais" \
<br>
last-lrm-refresh="1279241420" \
<br>
election-timeout="50s"
<br>
<br>
Ok when starting www01 (for the first time ) crm_mon show :
<br>
Online: [ filer2.connecting-nature.com filer1.connecting-nature.com
www01.connecting-nature.com ]
<br>
Resource Group: group_nfs
<br>
ip_nfs (ocf::heartbeat:IPaddr2): Started
filer1.connecting-nature.com
<br>
fs_nfs (ocf::heartbeat:Filesystem): Started
filer1.connecting-nature.com
<br>
nfs_server (lsb:nfs): Started filer1.connecting-nature.com
<br>
ip_failover (heartbeat:OVHfailover.py): Started
filer1.connecting-nature.com
<br>
Master/Slave Set: ms_drbd_nfs
<br>
Masters: [ filer1.connecting-nature.com ]
<br>
Slaves: [ filer2.connecting-nature.com ]
<br>
Clone Set: httpd-clone
<br>
Started: [ www01.connecting-nature.com ]
<br>
Stopped: [ httpd_server:1]
<br>
Master/Slave Set: ms_pureftpd_server
<br>
Slaves: [ www01.connecting-nature.com ]
<br>
Stopped: [ pureftpd_server:1 ]
<br>
<br>
Nice, everything's ok.
<br>
Now, i want to start www02, which is a vm cloned from www01. Only the
hostname and ipaddress changed !
<br>
Online: [ filer2.connecting-nature.com filer1.connecting-nature.com
www01.connecting-nature.com ]
<br>
OFFLINE: [ www02.connecting-nature.com ]
<br>
...
<br>
<br>
The machine correctly join the cluster, but always stay stopped...So i
did the test by stopping www02, stopping www01, and then starting
www02... It works, www02 goes online, but now, if i want to start
www01, it stays offline.
<br>
I can't start the the two node, each time one or the other will stay
offline.
<br>
If i shutdown the online node, after some time, the offline node will
come online, and when i restart the shooted node, it'll stay offline.
Sometimes services are started on the offline node but it remains
offline for corosync ! (it happens when i shutdown/launch www node in
random order to see what happens)
<br>
I tried without any services, that doesn't change anything.
<br>
<br>
Here are my corosync.conf :
<br>
# Please read the corosync.conf.5 manual page
<br>
compatibility: none
<br>
aisexec {
<br>
# Run as root - this is necessary to be able to manage
resources with Pacemaker
<br>
user: root
<br>
group: root
<br>
}
<br>
service {
<br>
# Load the Pacemaker Cluster Resource Manager
<br>
name: pacemaker
<br>
ver: 0
<br>
use_logd: yes
<br>
}
<br>
totem {
<br>
version: 2
<br>
# How long before declaring a token lost (ms)
<br>
token: 5000
<br>
hold: 40
<br>
# How many token retransmits before forming a new configuration
<br>
token_retransmits_before_loss_const: 30
<br>
# How long to wait for join messages in the membership protocol
(ms)
<br>
join: 1000
<br>
# How long to wait for consensus to be achieved before
<br>
# starting a new round of membership configuration (ms)
<br>
consensus: 7500
<br>
# Turn off the virtual synchrony filter
<br>
vsftype: none
<br>
# Number of messages that may be sent by one processor on
receipt of the token
<br>
max_messages: 20
<br>
# Limit generated nodeids to 31-bits (positive signed integers)
<br>
clear_node_high_bit: yes
<br>
# Enable encryption
<br>
secauth: off
<br>
threads: 0
<br>
# Optionally assign a fixed node id (integer)
<br>
nodeid: 30283707487
<br>
interface {
<br>
ringnumber: 0
<br>
bindnetaddr: 192.168.0.60
<br>
mcastport: 5406
<br>
mcastaddr: 225.0.0.1
<br>
}
<br>
}
<br>
logging {
<br>
fileline: on
<br>
to_stderr: yes
<br>
to_logfile: yes
<br>
to_syslog: yes
<br>
logfile: /var/log/corosync.log
<br>
debug: off
<br>
timestamp: on
<br>
logger_subsys {
<br>
subsys: AMF
<br>
debug: off
<br>
}
<br>
}
<br>
amf {
<br>
mode: disabled
<br>
}
<br>
<br>
the "bindnetaddress" change for every node, and the "nodeid" too (i
tried to put a fixed value to see if it was not a matter of auto
generated node id value, but it didn't change anything).
<br>
After some time, on the offline node, the only two processes running
are :
<br>
/usr/lib64/heartbeat/lrmd
<br>
/usr/lib64/heartbeat/pengine
<br>
<br>
(there should be
<br>
corosync
<br>
/usr/lib64/heartbeat/stonithd
<br>
/usr/lib64/heartbeat/cib
<br>
/usr/lib64/heartbeat/lrmd
<br>
/usr/lib64/heartbeat/attrd
<br>
/usr/lib64/heartbeat/pengine
<br>
/usr/lib64/heartbeat/crmd
<br>
)
<br>
<br>
I join the log here :
<br>
<br>
Jul 16 17:37:57 www01 corosync[25654]: [MAIN ] main.c:1307 Corosync
Cluster Engine ('1.2.3'): started and ready to provide service.
<br>
Jul 16 17:37:57 www01 corosync[25654]: [MAIN ] main.c:1308 Corosync
built-in features: nss rdma
<br>
Jul 16 17:37:57 www01 corosync[25654]: [MAIN ] main.c:1383
Successfully read main configuration file
'/etc/corosync/corosync.conf'.
<br>
Jul 16 17:37:57 www01 corosync[25654]: [TOTEM ] totemnet.c:204
Initializing transport (UDP/IP).
<br>
Jul 16 17:37:57 www01 corosync[25654]: [TOTEM ] totemudp.c:321
Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC
(mode 0).
<br>
Jul 16 17:37:58 www01 corosync[25654]: [TOTEM ] totemudp.c:1378 The
network interface [192.168.0.60] is now up.
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:330 info:
process_ais_conf: Reading configure
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:556 info:
config_find_init: Local handle: 2013064636357672962 for logging
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:582 info:
config_find_next: Processing additional logging options...
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:608 info:
get_config_opt: Found 'on' for option: debug
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:622 info:
get_config_opt: Defaulting to 'off' for option: to_file
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:608 info:
get_config_opt: Found 'yes' for option: to_syslog
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:622 info:
get_config_opt: Defaulting to 'daemon' for option: syslog_facility
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:556 info:
config_find_init: Local handle: 4730966301143465987 for service
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:582 info:
config_find_next: Processing additional service options...
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:622 info:
get_config_opt: Defaulting to 'pcmk' for option: clustername
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:608 info:
get_config_opt: Found 'yes' for option: use_logd
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:622 info:
get_config_opt: Defaulting to 'no' for option: use_mgmtd
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:554 info:
pcmk_startup: CRM: Initialized
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:555 Logging:
Initialized pcmk_startup
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:573 info:
pcmk_startup: Maximum core file size is: 18446744073709551615
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:594 info:
pcmk_startup: Service: 9
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:595 info:
pcmk_startup: Local hostname: www01.connecting-nature.com
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:510 info:
pcmk_update_nodeid: Local node id: 218936415
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:243 info:
update_member: Creating entry for node 218936415 born on 0
<br>
Jul 16 17:37:58 www01 stonith-ng: [25661]: WARN: Initializing
connection to logging daemon failed. Logging daemon may not be running
<br>
Jul 16 17:37:58 www01 stonith-ng: [25661]: info: Invoked:
/usr/lib64/heartbeat/stonithd
<br>
Jul 16 17:37:58 www01 stonith-ng: [25661]: info:
G_main_add_SignalHandler: Added signal handler for signal 17
<br>
Jul 16 17:37:58 www01 stonith-ng: [25661]: info: crm_cluster_connect:
Connecting to OpenAIS
<br>
Jul 16 17:37:58 www01 stonith-ng: [25661]: info: init_ais_connection:
Creating connection to our AIS plugin
<br>
Jul 16 17:37:58 www01 lrmd: [25663]: WARN: Initializing connection to
logging daemon failed. Logging daemon may not be running
<br>
Jul 16 17:37:58 www01 lrmd: [25663]: info: G_main_add_SignalHandler:
Added signal handler for signal 15
<br>
Jul 16 17:37:58 www01 cib: [25662]: WARN: Initializing connection to
logging daemon failed. Logging daemon may not be running
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: Invoked:
/usr/lib64/heartbeat/cib
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: G_main_add_TriggerHandler:
Added signal manual handler
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: G_main_add_SignalHandler:
Added signal handler for signal 17
<br>
Jul 16 17:37:58 www01 lrmd: [25663]: info: G_main_add_SignalHandler:
Added signal handler for signal 17
<br>
Jul 16 17:37:58 www01 attrd: [25664]: WARN: Initializing connection to
logging daemon failed. Logging daemon may not be running
<br>
Jul 16 17:37:58 www01 pengine: [25665]: WARN: Initializing connection
to logging daemon failed. Logging daemon may not be running
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:270 info:
update_member: 0x1259e10 Node 218936415 now known as
www01.connecting-nature.com (was: (null))
<br>
Jul 16 17:37:58 www01 crmd: [25666]: WARN: Initializing connection to
logging daemon failed. Logging daemon may not be running
<br>
Jul 16 17:37:58 www01 stonith-ng: [25661]: info: init_ais_connection:
AIS connection established
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: retrieveCib: Reading cluster
configuration from: /var/lib/heartbeat/crm/cib.xml (digest:
/var/lib/heartbeat/crm/cib.xml.sig)
<br>
Jul 16 17:37:58 www01 attrd: [25664]: info: Invoked:
/usr/lib64/heartbeat/attrd
<br>
Jul 16 17:37:58 www01 pengine: [25665]: info: Invoked:
/usr/lib64/heartbeat/pengine
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:286 info:
update_member: Node www01.connecting-nature.com now has 1 quorum votes
(was 0)
<br>
Jul 16 17:37:58 www01 crmd: [25666]: info: Invoked:
/usr/lib64/heartbeat/crmd
<br>
Jul 16 17:37:58 www01 cib: [25662]: WARN: retrieveCib: Cluster
configuration not found: /var/lib/heartbeat/crm/cib.xml
<br>
Jul 16 17:37:58 www01 stonith-ng: [25661]: info: get_ais_nodeid: Server
details: id=218936415 uname=www01.connecting-nature.com cname=pcmk
<br>
Jul 16 17:37:58 www01 lrmd: [25663]: info: G_main_add_SignalHandler:
Added signal handler for signal 10
<br>
Jul 16 17:37:58 www01 attrd: [25664]: info: main: Starting up
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:296 info:
update_member: Node 218936415/www01.connecting-nature.com is now:
member
<br>
Jul 16 17:37:58 www01 crmd: [25666]: info: main: CRM Hg Version:
972b9a5f68606f632893fceed658efa085062f55
<br>
Jul 16 17:37:58 www01 cib: [25662]: WARN: readCibXmlFile: Primary
configuration corrupt or unusable, trying backup...
<br>
Jul 16 17:37:58 www01 lrmd: [25663]: info: G_main_add_SignalHandler:
Added signal handler for signal 12
<br>
Jul 16 17:37:58 www01 attrd: [25664]: info: crm_cluster_connect:
Connecting to OpenAIS
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:132 info:
spawn_child: Forked child 25661 for process stonith-ng
<br>
Jul 16 17:37:58 www01 crmd: [25666]: info: crmd_init: Starting crmd
<br>
Jul 16 17:37:58 www01 stonith-ng: [25661]: info: crm_new_peer: Node
www01.connecting-nature.com now has id: 218936415
<br>
Jul 16 17:37:58 www01 attrd: [25664]: info: init_ais_connection:
Creating connection to our AIS plugin
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:132 info:
spawn_child: Forked child 25662 for process cib
<br>
Jul 16 17:37:58 www01 stonith-ng: [25661]: info: crm_new_peer: Node
218936415 is now known as www01.connecting-nature.com
<br>
Jul 16 17:37:58 www01 lrmd: [25663]: info: Started.
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:132 info:
spawn_child: Forked child 25663 for process lrmd
<br>
Jul 16 17:37:58 www01 attrd: [25664]: info: init_ais_connection: AIS
connection established
<br>
Jul 16 17:37:58 www01 cib: [25662]: WARN: readCibXmlFile: Continuing
with an empty configuration.
<br>
Jul 16 17:37:58 www01 stonith-ng: [25661]: info: main: Starting
stonith-ng mainloop
<br>
Jul 16 17:37:58 www01 pengine: [25665]: info: main: Starting pengine
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:132 info:
spawn_child: Forked child 25664 for process attrd
<br>
Jul 16 17:37:58 www01 attrd: [25664]: info: get_ais_nodeid: Server
details: id=218936415 uname=www01.connecting-nature.com cname=pcmk
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:132 info:
spawn_child: Forked child 25665 for process pengine
<br>
Jul 16 17:37:58 www01 attrd: [25664]: info: crm_new_peer: Node
www01.connecting-nature.com now has id: 218936415
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:132 info:
spawn_child: Forked child 25666 for process crmd
<br>
Jul 16 17:37:58 www01 attrd: [25664]: info: crm_new_peer: Node
218936415 is now known as www01.connecting-nature.com
<br>
Jul 16 17:37:58 www01 corosync[25654]: [SERV ] service.c:265 Service
engine loaded: Pacemaker Cluster Manager 1.1.1
<br>
Jul 16 17:37:58 www01 attrd: [25664]: info: main: Cluster connection
active
<br>
Jul 16 17:37:58 www01 corosync[25654]: [SERV ] service.c:265 Service
engine loaded: corosync extended virtual synchrony service
<br>
Jul 16 17:37:58 www01 crmd: [25666]: info: G_main_add_SignalHandler:
Added signal handler for signal 17
<br>
Jul 16 17:37:58 www01 attrd: [25664]: info: main: Accepting attribute
updates
<br>
Jul 16 17:37:58 www01 corosync[25654]: [SERV ] service.c:265 Service
engine loaded: corosync configuration service
<br>
Jul 16 17:37:58 www01 attrd: [25664]: info: main: Starting mainloop...
<br>
Jul 16 17:37:58 www01 corosync[25654]: [SERV ] service.c:265 Service
engine loaded: corosync cluster closed process group service v1.01
<br>
Jul 16 17:37:58 www01 corosync[25654]: [SERV ] service.c:265 Service
engine loaded: corosync cluster config database access v1.01
<br>
Jul 16 17:37:58 www01 corosync[25654]: [SERV ] service.c:265 Service
engine loaded: corosync profile loading service
<br>
Jul 16 17:37:58 www01 corosync[25654]: [SERV ] service.c:265 Service
engine loaded: corosync cluster quorum service v0.1
<br>
Jul 16 17:37:58 www01 corosync[25654]: [MAIN ] main.c:1220
Compatibility mode set to none. Using V2 of the synchronization
engine.
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: startCib: CIB Initialization
completed successfully
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: crm_cluster_connect:
Connecting to OpenAIS
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: init_ais_connection: Creating
connection to our AIS plugin
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: init_ais_connection: AIS
connection established
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: get_ais_nodeid: Server
details: id=218936415 uname=www01.connecting-nature.com cname=pcmk
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: crm_new_peer: Node
www01.connecting-nature.com now has id: 218936415
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: crm_new_peer: Node 218936415
is now known as www01.connecting-nature.com
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: cib_init: Starting cib
mainloop
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: ais_dispatch: Membership
145204: quorum still lost
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: crm_update_peer: Node
www01.connecting-nature.com: id=218936415 state=member (new) addr=r(0)
ip(192.168.0.60) (new) votes=1 (new) born=0 seen=145204
proc=00000000000000000000000000111312 (new)
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:699 notice:
pcmk_peer_update: Transitional membership event on ring 145204: memb=0,
new=0, lost=0
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:699 notice:
pcmk_peer_update: Stable membership event on ring 145204: memb=1,
new=1, lost=0
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:727 info:
pcmk_peer_update: NEW: www01.connecting-nature.com 218936415
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:743 info:
pcmk_peer_update: MEMB: www01.connecting-nature.com 218936415
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] utils.c:279 info:
update_member: Node www01.connecting-nature.com now has process list:
00000000000000000000000000111312 (1118994)
<br>
Jul 16 17:37:58 www01 corosync[25654]: [TOTEM ] totemsrp.c:1786 A
processor joined or left the membership and a new membership was
formed.
<br>
Jul 16 17:37:58 www01 corosync[25654]: [MAIN ] main.c:261 Completed
service synchronization, ready to provide service.
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:973 info:
pcmk_ipc: Recorded connection 0x1263600 for stonith-ng/25661
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:973 info:
pcmk_ipc: Recorded connection 0x1265f70 for attrd/25664
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:973 info:
pcmk_ipc: Recorded connection 0x1266a20 for cib/25662
<br>
Jul 16 17:37:58 www01 corosync[25654]: [pcmk ] plugin.c:982 info:
pcmk_ipc: Sending membership update 145204 to cib
<br>
Jul 16 17:37:58 www01 cib: [25670]: info: write_cib_contents: Wrote
version 0.0.0 of the CIB to disk (digest:
d73fab8054ae39ca7743184041837550)
<br>
Jul 16 17:37:58 www01 cib: [25670]: info: retrieveCib: Reading cluster
configuration from: /var/lib/heartbeat/crm/cib.zo9HRa (digest:
/var/lib/heartbeat/crm/cib.2sBGDK)
<br>
Jul 16 17:37:58 www01 cib: [25662]: info: Managed write_cib_contents
process 25670 exited with return code 0.
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: do_cib_control: CIB
connection established
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: crm_cluster_connect:
Connecting to OpenAIS
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: init_ais_connection:
Creating connection to our AIS plugin
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: init_ais_connection: AIS
connection established
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:973 info:
pcmk_ipc: Recorded connection 0x12659e0 for crmd/25666
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:982 info:
pcmk_ipc: Sending membership update 145204 to crmd
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: get_ais_nodeid: Server
details: id=218936415 uname=www01.connecting-nature.com cname=pcmk
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: crm_new_peer: Node
www01.connecting-nature.com now has id: 218936415
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: crm_new_peer: Node 218936415
is now known as www01.connecting-nature.com
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: do_ha_control: Connected to
the cluster
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:699 notice:
pcmk_peer_update: Transitional membership event on ring 145212: memb=1,
new=0, lost=0
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:710 info:
pcmk_peer_update: memb: www01.connecting-nature.com 218936415
<br>
Jul 16 17:37:59 www01 cib: [25662]: info: ais_dispatch: Membership
145212: quorum still lost
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:699 notice:
pcmk_peer_update: Stable membership event on ring 145212: memb=4,
new=3, lost=0
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] utils.c:243 info:
update_member: Creating entry for node 83929280 born on 145212
<br>
Jul 16 17:37:59 www01 cib: [25662]: info: crm_new_peer: Node
<null> now has id: 83929280
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] utils.c:296 info:
update_member: Node 83929280/unknown is now: member
<br>
Jul 16 17:37:59 www01 cib: [25662]: info: crm_update_peer: Node (null):
id=83929280 state=member (new) addr=r(0) ip(192.168.0.5) votes=0
born=0 seen=145212 proc=00000000000000000000000000000000
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:727 info:
pcmk_peer_update: NEW: .pending. 83929280
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: do_started: Delaying start,
CCM (0000000000100000) not connected
<br>
Jul 16 17:37:59 www01 cib: [25662]: info: crm_new_peer: Node
<null> now has id: 100706496
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] utils.c:243 info:
update_member: Creating entry for node 100706496 born on 145212
<br>
Jul 16 17:37:59 www01 cib: [25662]: info: crm_update_peer: Node (null):
id=100706496 state=member (new) addr=r(0) ip(192.168.0.6) votes=0
born=0 seen=145212 proc=00000000000000000000000000000000
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] utils.c:296 info:
update_member: Node 100706496/unknown is now: member
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:727 info:
pcmk_peer_update: NEW: .pending. 100706496
<br>
Jul 16 17:37:59 www01 cib: [25662]: info: crm_new_peer: Node
<null> now has id: 218437511
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: crmd_init: Starting crmd's
mainloop
<br>
Jul 16 17:37:59 www01 cib: [25662]: info: crm_update_peer: Node (null):
id=218437511 state=member (new) addr=r(0) ip(192.168.0.70) votes=0
born=0 seen=145212 proc=00000000000000000000000000000000
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] utils.c:243 info:
update_member: Creating entry for node 218437511 born on 145212
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: ais_dispatch: Membership
145204: quorum still lost
<br>
Jul 16 17:37:59 www01 cib: [25662]: notice: ais_dispatch: Membership
145212: quorum acquired
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] utils.c:296 info:
update_member: Node 218437511/unknown is now: member
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: crm_update_peer: Node
www01.connecting-nature.com: id=218936415 state=member (new) addr=r(0)
ip(192.168.0.60) (new) votes=1 (new) born=0 seen=145204
proc=00000000000000000000000000111312 (new)
<br>
Jul 16 17:37:59 www01 cib: [25662]: info: crm_get_peer: Node 100706496
is now known as filer2.connecting-nature.com
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:727 info:
pcmk_peer_update: NEW: .pending. 218437511
<br>
Jul 16 17:37:59 www01 cib: [25662]: info: crm_update_peer: Node
filer2.connecting-nature.com: id=100706496 state=member addr=r(0)
ip(192.168.0.6) votes=1 (new) born=15948 seen=145212
proc=00000000000000000000000000013312 (new)
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:743 info:
pcmk_peer_update: MEMB: .pending. 83929280
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:743 info:
pcmk_peer_update: MEMB: .pending. 100706496
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:743 info:
pcmk_peer_update: MEMB: www01.connecting-nature.com 218936415
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:743 info:
pcmk_peer_update: MEMB: .pending. 218437511
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:1288 info:
send_member_notification: Sending membership update 145212 to 2
children
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: do_started: Delaying start,
Config not read (0000000000000040)
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] utils.c:259 info:
update_member: 0x1259e10 Node 218936415 ((null)) born on: 145212
<br>
Jul 16 17:37:59 www01 corosync[25654]: [TOTEM ] totemsrp.c:1786 A
processor joined or left the membership and a new membership was
formed.
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] utils.c:259 info:
update_member: 0x1265450 Node 100706496 (filer2.connecting-nature.com)
born on: 15948
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] utils.c:270 info:
update_member: 0x1265450 Node 100706496 now known as
filer2.connecting-nature.com (was: (null))
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] utils.c:279 info:
update_member: Node filer2.connecting-nature.com now has process list:
00000000000000000000000000013312 (78610)
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] utils.c:286 info:
update_member: Node filer2.connecting-nature.com now has 1 quorum votes
(was 0)
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: config_query_callback:
Checking for expired actions every 900000ms
<br>
Jul 16 17:37:59 www01 corosync[25654]: [pcmk ] plugin.c:1288 info:
send_member_notification: Sending membership update 145212 to 2
children
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: config_query_callback:
Sending expected-votes=2 to corosync
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: ais_dispatch: Membership
145212: quorum still lost
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: crm_new_peer: Node
<null> now has id: 83929280
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: crm_update_peer: Node
(null): id=83929280 state=member (new) addr=r(0) ip(192.168.0.5)
votes=0 born=0 seen=145212 proc=00000000000000000000000000000000
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: crm_new_peer: Node
<null> now has id: 100706496
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: crm_update_peer: Node
(null): id=100706496 state=member (new) addr=r(0) ip(192.168.0.6)
votes=0 born=0 seen=145212 proc=00000000000000000000000000000000
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: crm_new_peer: Node
<null> now has id: 218437511
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: crm_update_peer: Node
(null): id=218437511 state=member (new) addr=r(0) ip(192.168.0.70)
votes=0 born=0 seen=145212 proc=00000000000000000000000000000000
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: do_started: The local CRM is
operational
<br>
Jul 16 17:37:59 www01 crmd: [25666]: info: do_state_transition: State
transition S_STARTING -> S_PENDING [ input=I_PENDING
cause=C_FSA_INTERNAL origin=do_started ]
<br>
Jul 16 17:38:00 www01 crmd: [25666]: notice: ais_dispatch: Membership
145212: quorum acquired
<br>
Jul 16 17:38:00 www01 crmd: [25666]: info: crm_get_peer: Node 100706496
is now known as filer2.connecting-nature.com
<br>
Jul 16 17:38:00 www01 crmd: [25666]: info: crm_update_peer: Node
filer2.connecting-nature.com: id=100706496 state=member addr=r(0)
ip(192.168.0.6) votes=1 (new) born=15948 seen=145212
proc=00000000000000000000000000013312 (new)
<br>
Jul 16 17:38:00 www01 crmd: [25666]: info: ais_dispatch: Membership
145212: quorum retained
<br>
Jul 16 17:38:00 www01 crmd: [25666]: info: te_connect_stonith:
Attempting connection to fencing daemon...
<br>
Jul 16 17:38:01 www01 crmd: [25666]: info: te_connect_stonith:
Connected
<br>
Jul 16 17:38:03 www01 attrd: [25664]: info: cib_connect: Connected to
the CIB after 1 signon attempts
<br>
Jul 16 17:38:03 www01 attrd: [25664]: info: cib_connect: Sending full
refresh
<br>
Jul 16 17:38:08 www01 cib: [25662]: info: ais_dispatch: Membership
145212: quorum retained
<br>
Jul 16 17:38:08 www01 crmd: [25666]: info: ais_dispatch: Membership
145212: quorum retained
<br>
Jul 16 17:38:08 www01 cib: [25662]: info: crm_get_peer: Node 218437511
is now known as www02.connecting-nature.com
<br>
Jul 16 17:38:08 www01 crmd: [25666]: info: crm_get_peer: Node 218437511
is now known as www02.connecting-nature.com
<br>
Jul 16 17:38:08 www01 cib: [25662]: info: crm_update_peer: Node
www02.connecting-nature.com: id=218437511 state=member addr=r(0)
ip(192.168.0.70) votes=1 (new) born=145144 seen=145212
proc=00000000000000000000000000111312 (new)
<br>
Jul 16 17:38:08 www01 crmd: [25666]: info: crm_update_peer: Node
www02.connecting-nature.com: id=218437511 state=member addr=r(0)
ip(192.168.0.70) votes=1 (new) born=145144 seen=145212
proc=00000000000000000000000000111312 (new)
<br>
Jul 16 17:38:08 www01 cib: [25662]: info: ais_dispatch: Membership
145212: quorum retained
<br>
Jul 16 17:38:08 www01 cib: [25662]: info: crm_get_peer: Node 83929280
is now known as filer1.connecting-nature.com
<br>
Jul 16 17:38:08 www01 cib: [25662]: info: crm_update_peer: Node
filer1.connecting-nature.com: id=83929280 state=member addr=r(0)
ip(192.168.0.5) votes=1 (new) born=15948 seen=145212
proc=00000000000000000000000000013312 (new)
<br>
Jul 16 17:38:08 www01 cib: [25662]: info: cib_process_diff: Diff
0.755.35 -> 0.755.36 not applied to 0.0.0: current "epoch" is less
than required
<br>
Jul 16 17:38:08 www01 cib: [25662]: info: cib_server_process_diff:
Requesting re-sync from peer
<br>
Jul 16 17:38:08 www01 cib: [25662]: WARN: cib_diff_notify: Local-only
Change (client:crmd, call: 5456): 0.0.0 (Application of an update diff
failed, requesting a full refresh)
<br>
Jul 16 17:38:08 www01 crmd: [25666]: info: ais_dispatch: Membership
145212: quorum retained
<br>
Jul 16 17:38:08 www01 crmd: [25666]: info: crm_get_peer: Node 83929280
is now known as filer1.connecting-nature.com
<br>
Jul 16 17:38:08 www01 crmd: [25666]: info: crm_update_peer: Node
filer1.connecting-nature.com: id=83929280 state=member addr=r(0)
ip(192.168.0.5) votes=1 (new) born=15948 seen=145212
proc=00000000000000000000000000013312 (new)
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] utils.c:259 info:
update_member: 0x1265700 Node 218437511 (www02.connecting-nature.com)
born on: 145144
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] utils.c:270 info:
update_member: 0x1265700 Node 218437511 now known as
www02.connecting-nature.com (was: (null))
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] utils.c:279 info:
update_member: Node www02.connecting-nature.com now has process list:
00000000000000000000000000111312 (1118994)
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] utils.c:286 info:
update_member: Node www02.connecting-nature.com now has 1 quorum votes
(was 0)
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:317 info:
update_expected_votes: Expected quorum votes 2 -> 3
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:1288 info:
send_member_notification: Sending membership update 145212 to 2
children
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] utils.c:259 info:
update_member: 0x1265140 Node 83929280 (filer1.connecting-nature.com)
born on: 15948
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] utils.c:270 info:
update_member: 0x1265140 Node 83929280 now known as
filer1.connecting-nature.com (was: (null))
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] utils.c:279 info:
update_member: Node filer1.connecting-nature.com now has process list:
00000000000000000000000000013312 (78610)
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] utils.c:286 info:
update_member: Node filer1.connecting-nature.com now has 1 quorum votes
(was 0)
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:317 info:
update_expected_votes: Expected quorum votes 3 -> 4
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:1288 info:
send_member_notification: Sending membership update 145212 to 2
children
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:699 notice:
pcmk_peer_update: Transitional membership event on ring 145224: memb=3,
new=0, lost=1
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:710 info:
pcmk_peer_update: memb: filer2.connecting-nature.com 100706496
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:710 info:
pcmk_peer_update: memb: www01.connecting-nature.com 218936415
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:710 info:
pcmk_peer_update: memb: www02.connecting-nature.com 218437511
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:715 info:
pcmk_peer_update: lost: filer1.connecting-nature.com 83929280
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:699 notice:
pcmk_peer_update: Stable membership event on ring 145224: memb=4,
new=1, lost=0
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:727 info:
pcmk_peer_update: NEW: filer1.connecting-nature.com 83929280
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:743 info:
pcmk_peer_update: MEMB: filer1.connecting-nature.com 83929280
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:743 info:
pcmk_peer_update: MEMB: filer2.connecting-nature.com 100706496
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:743 info:
pcmk_peer_update: MEMB: www01.connecting-nature.com 218936415
<br>
Jul 16 17:38:08 www01 corosync[25654]: [pcmk ] plugin.c:743 info:
pcmk_peer_update: MEMB: www02.connecting-nature.com 218437511
<br>
Jul 16 17:38:08 www01 corosync[25654]: [TOTEM ] totemsrp.c:1786 A
processor joined or left the membership and a new membership was
formed.
<br>
Jul 16 17:38:11 www01 stonith-ng: [25661]: ERROR: ais_dispatch:
Receiving message body failed: (2) Library error: Resource temporarily
unavailable (11)
<br>
Jul 16 17:38:11 www01 cib: [25662]: ERROR: ais_dispatch: Receiving
message body failed: (2) Library error: Resource temporarily
unavailable (11)
<br>
Jul 16 17:38:11 www01 stonith-ng: [25661]: ERROR: ais_dispatch: AIS
connection failed
<br>
Jul 16 17:38:11 www01 cib: [25662]: ERROR: ais_dispatch: AIS connection
failed
<br>
Jul 16 17:38:11 www01 cib: [25662]: ERROR: cib_ais_destroy: AIS
connection terminated
<br>
Jul 16 17:38:11 www01 stonith-ng: [25661]: ERROR:
stonith_peer_ais_destroy: AIS connection terminated
<br>
Jul 16 17:38:11 www01 attrd: [25664]: ERROR: ais_dispatch: Receiving
message body failed: (2) Library error: Resource temporarily
unavailable (11)
<br>
Jul 16 17:38:11 www01 attrd: [25664]: ERROR: ais_dispatch: AIS
connection failed
<br>
Jul 16 17:38:11 www01 attrd: [25664]: CRIT: attrd_ais_destroy: Lost
connection to OpenAIS service!
<br>
Jul 16 17:38:11 www01 attrd: [25664]: info: main: Exiting...
<br>
Jul 16 17:38:11 www01 crmd: [25666]: info: cib_native_msgready: Lost
connection to the CIB service [25662].
<br>
Jul 16 17:38:11 www01 crmd: [25666]: CRIT: cib_native_dispatch: Lost
connection to the CIB service [25662/callback].
<br>
Jul 16 17:38:11 www01 crmd: [25666]: CRIT: cib_native_dispatch: Lost
connection to the CIB service [25662/command].
<br>
Jul 16 17:38:11 www01 attrd: [25664]: ERROR:
attrd_cib_connection_destroy: Connection to the CIB terminated...
<br>
Jul 16 17:38:11 www01 crmd: [25666]: ERROR:
crmd_cib_connection_destroy: Connection to the CIB terminated...
<br>
Jul 16 17:38:11 www01 crmd: [25666]: ERROR: ais_dispatch: Receiving
message body failed: (2) Library error: Invalid argument (22)
<br>
Jul 16 17:38:11 www01 crmd: [25666]: ERROR: ais_dispatch: AIS
connection failed
<br>
Jul 16 17:38:11 www01 crmd: [25666]: ERROR: crm_ais_destroy: AIS
connection terminated
<br>
<br>
sorry for this looooooonnnggg post, but i tried to be the most
exhaustive
<br>
<br>
<br>
Thanks for your help if possible
<br>
Guillaume
<br>
</div>
</body>
</html>