[ClusterLabs] stonith-ng - performing action 'monitor' timed out with signal 15
Marco Marino
marino.mrc at gmail.com
Tue Sep 3 04:09:26 EDT 2019
Hi, I have a problem with fencing on a two node cluster. It seems that
randomly the cluster cannot complete monitor operation for fence devices.
In log I see:
crmd[8206]: error: Result of monitor operation for fence-node2 on
ld2.mydomain.it: Timed Out
As attachment there is
- /var/log/messages for node1 (only the important part)
- /var/log/messages for node2 (only the important part) <-- Problem starts
here
- pcs status
- pcs stonith show (for both fence devices)
I think it could be a timeout problem, so how can I see timeout value for
monitor operation in stonith devices?
Please, someone can help me with this problem?
Furthermore, how can I fix the state of fence devices without downtime?
Thank you
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20190903/9ad440b4/attachment-0001.html>
-------------- next part --------------
############PCS STATUS########################
root at ld1 ~]# pcs status
Cluster name: ldcluster
Stack: corosync
Current DC: ld1.mydomain.it (version 1.1.19-8.el7_6.4-c3c624ea3d) - partition with quorum
Last updated: Tue Sep 3 09:37:27 2019
Last change: Thu Jul 4 21:36:07 2019 by root via cibadmin on ld1.mydomain.it
2 nodes configured
10 resources configured
Online: [ ld1.mydomain.it ld2.mydomain.it ]
Full list of resources:
fence-node1 (stonith:fence_ipmilan): Stopped
fence-node2 (stonith:fence_ipmilan): Stopped
Master/Slave Set: DrbdResClone [DrbdRes]
Masters: [ ld1.mydomain.it ]
Slaves: [ ld2.mydomain.it ]
HALVM (ocf::heartbeat:LVM): Started ld1.mydomain.it
PgsqlFs (ocf::heartbeat:Filesystem): Started ld1.mydomain.it
PostgresqlD (systemd:postgresql-9.6.service): Started ld1.mydomain.it
LegaldocapiD (systemd:legaldocapi.service): Started ld1.mydomain.it
PublicVIP (ocf::heartbeat:IPaddr2): Started ld1.mydomain.it
DefaultRoute (ocf::heartbeat:Route): Started ld1.mydomain.it
Failed Actions:
* fence-node1_start_0 on ld1.mydomain.it 'unknown error' (1): call=221, status=Timed Out, exitreason='',
last-rc-change='Wed Aug 21 12:49:00 2019', queued=0ms, exec=20006ms
* fence-node2_start_0 on ld1.mydomain.it 'unknown error' (1): call=222, status=Timed Out, exitreason='',
last-rc-change='Wed Aug 21 12:49:00 2019', queued=1ms, exec=20013ms
* fence-node1_start_0 on ld2.mydomain.it 'unknown error' (1): call=182, status=Timed Out, exitreason='',
last-rc-change='Wed Aug 21 14:26:09 2019', queued=0ms, exec=20006ms
* fence-node2_start_0 on ld2.mydomain.it 'unknown error' (1): call=176, status=Timed Out, exitreason='',
last-rc-change='Wed Aug 21 12:48:40 2019', queued=1ms, exec=20008ms
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
[root at ld1 ~]#
########################STONITH SHOW###########################################
[root at ld1 ~]# pcs stonith show fence-node1
Resource: fence-node1 (class=stonith type=fence_ipmilan)
Attributes: ipaddr=192.168.254.250 lanplus=1 login=root passwd=XXXXXXX pcmk_host_check=static-list pcmk_host_list=ld1.mydomain.it
Operations: monitor interval=60s (fence-node1-monitor-interval-60s)
[root at ld1 ~]# pcs stonith show fence-node2
Resource: fence-node2 (class=stonith type=fence_ipmilan)
Attributes: ipaddr=192.168.254.251 lanplus=1 login=root passwd=XXXXXXXX pcmk_host_check=static-list pcmk_host_list=ld2.mydomain.it delay=12
Operations: monitor interval=60s (fence-node2-monitor-interval-60s)
[root at ld1 ~]#
###########################NODE 2 /var/log/messages##############################
Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: Child process 46006 performing action 'monitor' timed out with signal 15
Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: Operation 'monitor' [46006] for device 'fence-node2' returned: -62 (Timer expired)
Aug 21 12:48:40 ld2 crmd[8206]: error: Result of monitor operation for fence-node2 on ld2.mydomain.it: Timed Out
Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:48:40 ld2 crmd[8206]: notice: Result of stop operation for fence-node2 on ld2.mydomain.it: 0 (ok)
Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:48:40 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:48:59 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: Child process 46053 performing action 'monitor' timed out with signal 15
Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: Operation 'monitor' [46053] for device 'fence-node2' returned: -62 (Timer expired)
Aug 21 12:49:00 ld2 crmd[8206]: error: Result of start operation for fence-node2 on ld2.mydomain.it: Timed Out
Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld2 crmd[8206]: notice: Result of stop operation for fence-node2 on ld2.mydomain.it: 0 (ok)
Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:20 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:31 ld2 fence_ipmilan: Failed: Unable to obtain correct plug status or plug is not available
Aug 21 12:49:31 ld2 stonith-ng[8202]: warning: fence_ipmilan[46088] stderr: [ 2019-08-21 12:49:31,216 ERROR: Failed: Unable to obtain correct plug status or plug is not available ]
Aug 21 12:49:31 ld2 stonith-ng[8202]: warning: fence_ipmilan[46088] stderr: [ ]
Aug 21 12:49:31 ld2 stonith-ng[8202]: warning: fence_ipmilan[46088] stderr: [ ]
Aug 21 12:49:32 ld2 crmd[8206]: notice: Result of start operation for fence-node1 on ld2.mydomain.it: 0 (ok)
Aug 21 12:49:32 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:32 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:32 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:50:01 ld2 systemd: Started Session 28060 of user root.
Aug 21 13:00:01 ld2 systemd: Started Session 28061 of user root.
Aug 21 13:01:01 ld2 systemd: Started Session 28062 of user root.
Aug 21 13:10:01 ld2 systemd: Started Session 28063 of user root.
Aug 21 13:20:01 ld2 systemd: Started Session 28064 of user root.
Aug 21 13:30:01 ld2 systemd: Started Session 28065 of user root.
Aug 21 13:40:01 ld2 systemd: Started Session 28066 of user root.
Aug 21 13:50:01 ld2 systemd: Started Session 28067 of user root.
Aug 21 14:00:01 ld2 systemd: Started Session 28068 of user root.
Aug 21 14:01:01 ld2 systemd: Started Session 28069 of user root.
Aug 21 14:10:01 ld2 systemd: Started Session 28070 of user root.
Aug 21 14:20:01 ld2 systemd: Started Session 28071 of user root.
Aug 21 14:26:08 ld2 stonith-ng[8202]: notice: Child process 4835 performing action 'monitor' timed out with signal 15
Aug 21 14:26:08 ld2 stonith-ng[8202]: notice: Operation 'monitor' [4835] for device 'fence-node1' returned: -62 (Timer expired)
Aug 21 14:26:08 ld2 crmd[8206]: error: Result of monitor operation for fence-node1 on ld2.mydomain.it: Timed Out
Aug 21 14:26:08 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 14:26:09 ld2 crmd[8206]: notice: Result of stop operation for fence-node1 on ld2.mydomain.it: 0 (ok)
Aug 21 14:26:09 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 14:26:09 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 14:26:09 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 14:26:29 ld2 stonith-ng[8202]: notice: Child process 4892 performing action 'monitor' timed out with signal 15
Aug 21 14:26:29 ld2 stonith-ng[8202]: notice: Operation 'monitor' [4892] for device 'fence-node1' returned: -62 (Timer expired)
Aug 21 14:26:29 ld2 crmd[8206]: error: Result of start operation for fence-node1 on ld2.mydomain.it: Timed Out
Aug 21 14:26:29 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 14:26:29 ld2 crmd[8206]: notice: Result of stop operation for fence-node1 on ld2.mydomain.it: 0 (ok)
Aug 21 14:26:29 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
Aug 21 14:26:29 ld2 stonith-ng[8202]: notice: On loss of CCM Quorum: Ignore
###########################NODE 1 /var/log/messages##############################
Aug 21 12:48:40 ld1 crmd[8457]: notice: State transition S_IDLE -> S_POLICY_ENGINE
Aug 21 12:48:40 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:48:40 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:48:40 ld1 pengine[8456]: warning: Processing failed monitor of fence-node2 on ld2.mydomain.it: unknown error
Aug 21 12:48:40 ld1 pengine[8456]: notice: * Recover fence-node2 ( ld2.mydomain.it )
Aug 21 12:48:40 ld1 pengine[8456]: notice: Calculated transition 15937, saving inputs in /var/lib/pacemaker/pengine/pe-input-95.bz2
Aug 21 12:48:40 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:48:40 ld1 pengine[8456]: warning: Processing failed monitor of fence-node2 on ld2.mydomain.it: unknown error
Aug 21 12:48:40 ld1 pengine[8456]: notice: * Recover fence-node2 ( ld2.mydomain.it )
Aug 21 12:48:40 ld1 pengine[8456]: notice: Calculated transition 15938, saving inputs in /var/lib/pacemaker/pengine/pe-input-96.bz2
Aug 21 12:48:40 ld1 crmd[8457]: notice: Initiating stop operation fence-node2_stop_0 on ld2.mydomain.it
Aug 21 12:48:40 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:48:40 ld1 crmd[8457]: notice: Initiating start operation fence-node2_start_0 on ld2.mydomain.it
Aug 21 12:48:40 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:48:40 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:48:43 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:48:46 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:48:49 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:48:52 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:48:55 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:48:58 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:48:59 ld1 stonith-ng[8453]: notice: Child process 13446 performing action 'monitor' timed out with signal 15
Aug 21 12:48:59 ld1 stonith-ng[8453]: notice: Operation 'monitor' [13446] for device 'fence-node1' returned: -62 (Timer expired)
Aug 21 12:48:59 ld1 crmd[8457]: error: Result of monitor operation for fence-node1 on ld1.mydomain.it: Timed Out
Aug 21 12:48:59 ld1 crmd[8457]: notice: Transition aborted by operation fence-node1_monitor_60000 'create' on ld1.mydomain.it: Old event
Aug 21 12:48:59 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld1 crmd[8457]: warning: Action 14 (fence-node2_start_0) on ld2.mydomain.it failed (target: 0 vs. rc: 1): Error
Aug 21 12:49:00 ld1 crmd[8457]: notice: Transition 15938 (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=1, Source=/var/lib/pacemaker/pengine/pe-input-96.bz2): Complete
Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed monitor of fence-node1 on ld1.mydomain.it: unknown error
Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error
Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error
Aug 21 12:49:00 ld1 pengine[8456]: notice: * Recover fence-node1 ( ld1.mydomain.it )
Aug 21 12:49:00 ld1 pengine[8456]: notice: * Recover fence-node2 ( ld2.mydomain.it )
Aug 21 12:49:00 ld1 pengine[8456]: notice: Calculated transition 15939, saving inputs in /var/lib/pacemaker/pengine/pe-input-97.bz2
Aug 21 12:49:00 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed monitor of fence-node1 on ld1.mydomain.it: unknown error
Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error
Aug 21 12:49:00 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error
Aug 21 12:49:00 ld1 pengine[8456]: warning: Forcing fence-node2 away from ld2.mydomain.it after 1000000 failures (max=1000000)
Aug 21 12:49:00 ld1 pengine[8456]: notice: * Recover fence-node1 ( ld1.mydomain.it )
Aug 21 12:49:00 ld1 pengine[8456]: notice: * Recover fence-node2 ( ld2.mydomain.it -> ld1.mydomain.it )
Aug 21 12:49:00 ld1 pengine[8456]: notice: Calculated transition 15940, saving inputs in /var/lib/pacemaker/pengine/pe-input-98.bz2
Aug 21 12:49:00 ld1 crmd[8457]: notice: Initiating stop operation fence-node1_stop_0 locally on ld1.mydomain.it
Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld1 crmd[8457]: notice: Initiating stop operation fence-node2_stop_0 on ld2.mydomain.it
Aug 21 12:49:00 ld1 crmd[8457]: notice: Result of stop operation for fence-node1 on ld1.mydomain.it: 0 (ok)
Aug 21 12:49:00 ld1 crmd[8457]: notice: Initiating start operation fence-node1_start_0 locally on ld1.mydomain.it
Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld1 crmd[8457]: notice: Initiating start operation fence-node2_start_0 locally on ld1.mydomain.it
Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:00 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:01 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:49:04 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:49:07 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:49:10 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:49:13 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:49:16 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:49:19 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:49:20 ld1 stonith-ng[8453]: notice: Child process 13654 performing action 'monitor' timed out with signal 15
Aug 21 12:49:20 ld1 stonith-ng[8453]: notice: Operation 'monitor' [13654] for device 'fence-node1' returned: -62 (Timer expired)
Aug 21 12:49:20 ld1 crmd[8457]: error: Result of start operation for fence-node1 on ld1.mydomain.it: Timed Out
Aug 21 12:49:20 ld1 crmd[8457]: warning: Action 12 (fence-node1_start_0) on ld1.mydomain.it failed (target: 0 vs. rc: 1): Error
Aug 21 12:49:20 ld1 crmd[8457]: notice: Transition aborted by operation fence-node1_start_0 'modify' on ld1.mydomain.it: Event failed
Aug 21 12:49:20 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:20 ld1 crmd[8457]: notice: Transition aborted by status-1-fail-count-fence-node1.start_0 doing create fail-count-fence-node1#start_0=INFINITY: Transient attribute change
Aug 21 12:49:20 ld1 stonith-ng[8453]: notice: Child process 13656 performing action 'monitor' timed out with signal 15
Aug 21 12:49:20 ld1 stonith-ng[8453]: notice: Operation 'monitor' [13656] for device 'fence-node2' returned: -62 (Timer expired)
Aug 21 12:49:21 ld1 crmd[8457]: error: Result of start operation for fence-node2 on ld1.mydomain.it: Timed Out
Aug 21 12:49:21 ld1 crmd[8457]: warning: Action 13 (fence-node2_start_0) on ld1.mydomain.it failed (target: 0 vs. rc: 1): Error
Aug 21 12:49:21 ld1 crmd[8457]: notice: Transition 15940 (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=2, Source=/var/lib/pacemaker/pengine/pe-input-98.bz2): Complete
Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node1 on ld1.mydomain.it: unknown error
Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node1 on ld1.mydomain.it: unknown error
Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld1.mydomain.it: unknown error
Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld1.mydomain.it: unknown error
Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error
Aug 21 12:49:21 ld1 pengine[8456]: warning: Forcing fence-node1 away from ld1.mydomain.it after 1000000 failures (max=1000000)
Aug 21 12:49:21 ld1 pengine[8456]: warning: Forcing fence-node2 away from ld2.mydomain.it after 1000000 failures (max=1000000)
Aug 21 12:49:21 ld1 pengine[8456]: notice: * Recover fence-node1 ( ld1.mydomain.it -> ld2.mydomain.it )
Aug 21 12:49:21 ld1 pengine[8456]: notice: * Recover fence-node2 ( ld1.mydomain.it )
Aug 21 12:49:21 ld1 pengine[8456]: notice: Calculated transition 15941, saving inputs in /var/lib/pacemaker/pengine/pe-input-99.bz2
Aug 21 12:49:21 ld1 pengine[8456]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node1 on ld1.mydomain.it: unknown error
Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node1 on ld1.mydomain.it: unknown error
Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld1.mydomain.it: unknown error
Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld1.mydomain.it: unknown error
Aug 21 12:49:21 ld1 pengine[8456]: warning: Processing failed start of fence-node2 on ld2.mydomain.it: unknown error
Aug 21 12:49:21 ld1 pengine[8456]: warning: Forcing fence-node1 away from ld1.mydomain.it after 1000000 failures (max=1000000)
Aug 21 12:49:21 ld1 pengine[8456]: warning: Forcing fence-node2 away from ld1.mydomain.it after 1000000 failures (max=1000000)
Aug 21 12:49:21 ld1 pengine[8456]: warning: Forcing fence-node2 away from ld2.mydomain.it after 1000000 failures (max=1000000)
Aug 21 12:49:21 ld1 pengine[8456]: notice: * Recover fence-node1 ( ld1.mydomain.it -> ld2.mydomain.it )
Aug 21 12:49:21 ld1 pengine[8456]: notice: * Stop fence-node2 ( ld1.mydomain.it ) due to node availability
Aug 21 12:49:21 ld1 pengine[8456]: notice: Calculated transition 15942, saving inputs in /var/lib/pacemaker/pengine/pe-input-100.bz2
Aug 21 12:49:21 ld1 crmd[8457]: notice: Initiating stop operation fence-node1_stop_0 locally on ld1.mydomain.it
Aug 21 12:49:21 ld1 crmd[8457]: notice: Initiating stop operation fence-node2_stop_0 locally on ld1.mydomain.it
Aug 21 12:49:21 ld1 crmd[8457]: notice: Result of stop operation for fence-node1 on ld1.mydomain.it: 0 (ok)
Aug 21 12:49:21 ld1 crmd[8457]: notice: Result of stop operation for fence-node2 on ld1.mydomain.it: 0 (ok)
Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld1 crmd[8457]: notice: Initiating start operation fence-node1_start_0 on ld2.mydomain.it
Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:21 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:22 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:49:25 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:49:28 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:49:31 ld1 snmpd[20884]: refused smux peer: oid SNMPv2-SMI::enterprises.674.10892.1, descr Systems Management SNMP MIB Plug-in Manager
Aug 21 12:49:32 ld1 crmd[8457]: notice: Initiating monitor operation fence-node1_monitor_60000 on ld2.mydomain.it
Aug 21 12:49:32 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:32 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
Aug 21 12:49:32 ld1 crmd[8457]: notice: Transition 15942 (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-100.bz2): Complete
Aug 21 12:49:32 ld1 crmd[8457]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE
Aug 21 12:49:32 ld1 stonith-ng[8453]: notice: On loss of CCM Quorum: Ignore
More information about the Users
mailing list