[ClusterLabs] 2 node cluster dlm/clvm trouble

Patrick Whitney pwhitney at luminoso.com
Thu Sep 6 10:36:03 EDT 2018


Good Morning Everyone,

I'm hoping someone with more experience with corosync and pacemaker can see
what I am doing wrong.

I've got a test setup of 2 nodes, with dlm and clvm setup as clones, and
using fence_scsi as my fencing agent.

I've got it to the point where the cluster is up, and reports it is happy.
I then began testing fencing.   When issuing 'pcs stonith fence' it appears
to work; that is, the scsi reservation is pulled and the output of 'pcs
status' looks sane, and I'm able to access resources on the un-fenced node.

Things go awry when I shutdown (init 0) the fenced node... my unfenced node
decides to fence itself (which looks like it was initiated by dlm due to an
abandoned lockspace).

I suspect this is due to misconfiguration, since I'm new to the toolset,
but I'm not quite sure what I need to change.

Any and all input is appreciated!

Below is a chronology of events; my corosync config and cib.xml; command
output; and annotated logs.

Again, any hints, suggestions, wild guesses, or premonitions are welcomed
-- I'm stuck!   Please let me know if there is additional information which
would be helpful.

Many thanks,
-Patrick W.

Sep  6 08:54:14  -- Cluster is up and running; UI reports everything
                               healthy.

Sep  6 08:55:44  -- 'pcs stonith fence' called against node 1
(coro-test-1);
                               UI reports everything as expected -- that
is, resources show only running on unfenced node and they're available.
                               Oddly, although the UI says dlm is stopped
on fenced node, the dlm_controld is still running.

Sep  6 09:03:38  -- node 1 is shutdown, and node 2 falls to pieces.
                               - First, corosync sees lost member -- seems
like this is appropriate, to me.
                               - Next, dlm_controld calls to fence
everything
                               - stonith-ng tries to fence node 1 (but its
already fenced!)
                               - dlm closes connection to "node 2" (does
dlm "nodes" map to cluster nodes? I'm not sure they do)
                               - clvmd dlm lockspace is now abandoned;
cluster attempts to fence the remaining node
                                 (But can't because scsi_fence doesn't work
like that).

***
******   -- Configuration --
***
root at coro-test-2:~# pcs --version
0.9.149
root at coro-test-2:~# pacemakerd --version
Pacemaker 1.1.14
Written by Andrew Beekhof
root at coro-test-2:~# corosync -v
Corosync Cluster Engine, version '2.3.5'
Copyright (c) 2006-2009 Red Hat, Inc.
root at coro-test-2:~#

#
##  COROSYNC
#
root at coro-test-1:~# more /etc/corosync/corosync.conf
totem {
    version: 2
    secauth: off
    cluster_name: my_first_cluster
    transport: udpu
}

nodelist {
    node {
        ring0_addr: coro-test-1
        nodeid: 1
    }

    node {
        ring0_addr: coro-test-2
        nodeid: 2
    }
}

quorum {
    provider: corosync_votequorum
    two_node: 1
}

logging {
    to_logfile: yes
    logfile: /var/log/corosync/corosync.log
    to_syslog: yes
}

##
###  PACEMAKER CIB.XML
##
root at coro-test-1:/var/lib/pacemaker/cib# cat cib.xml
<cib crm_feature_set="3.0.10" validate-with="pacemaker-2.4" epoch="30"
num_updates="0" admin_epoch="0" cib-last-written="Thu Sep  6 10:12:46 2018"
update-origin="coro-test-2" update-client="cibadmin" update-user="root"
have-quorum="1" dc-uuid="1">
  <configuration>
    <crm_config>
      <cluster_property_set id="cib-bootstrap-options">
        <nvpair id="cib-bootstrap-options-have-watchdog"
name="have-watchdog" value="false"/>
        <nvpair id="cib-bootstrap-options-dc-version" name="dc-version"
value="1.1.14-70404b0"/>
        <nvpair id="cib-bootstrap-options-cluster-infrastructure"
name="cluster-infrastructure" value="corosync"/>
        <nvpair id="cib-bootstrap-options-cluster-name" name="cluster-name"
value="my_first_cluster"/>
        <nvpair id="cib-bootstrap-options-no-quorum-policy"
name="no-quorum-policy" value="ignore"/>
        <nvpair id="cib-bootstrap-options-stonith-enabled"
name="stonith-enabled" value="true"/>
        <nvpair id="cib-bootstrap-options-stonith-action"
name="stonith-action" value="off"/>
      </cluster_property_set>
    </crm_config>
    <nodes>
      <node id="1" uname="coro-test-1"/>
      <node id="2" uname="coro-test-2"/>
    </nodes>
    <resources>
      <clone id="dlm-clone">
        <primitive class="ocf" id="dlm" provider="pacemaker"
type="controld">
          <instance_attributes id="dlm-instance_attributes"/>
          <operations>
            <op id="dlm-start-interval-0s" interval="0s" name="start"
timeout="90"/>
            <op id="dlm-stop-interval-0s" interval="0s" name="stop"
timeout="100"/>
            <op id="dlm-monitor-interval-30s" interval="30s" name="monitor"
on-fail="fence"/>
          </operations>
        </primitive>
        <meta_attributes id="dlm-clone-meta_attributes">
          <nvpair id="dlm-clone-meta_attributes-interleave"
name="interleave" value="true"/>
          <nvpair id="dlm-clone-meta_attributes-ordered" name="ordered"
value="true"/>
        </meta_attributes>
      </clone>
      <clone id="clvmd-clone">
        <primitive class="ocf" id="clvmd" provider="heartbeat" type="clvm">
          <instance_attributes id="clvmd-instance_attributes"/>
          <operations>
            <op id="clvmd-start-interval-0s" interval="0s" name="start"
timeout="90"/>
            <op id="clvmd-stop-interval-0s" interval="0s" name="stop"
timeout="90"/>
            <op id="clvmd-monitor-interval-30s" interval="30s"
name="monitor" on-fail="fence"/>
          </operations>
        </primitive>
        <meta_attributes id="clvmd-clone-meta_attributes">
          <nvpair id="clvmd-clone-meta_attributes-interleave"
name="interleave" value="true"/>
          <nvpair id="clvmd-clone-meta_attributes-ordered" name="ordered"
value="true"/>
        </meta_attributes>
      </clone>
      <primitive class="stonith" id="scsi-shooter" type="fence_scsi">
        <instance_attributes id="scsi-shooter-instance_attributes">
          <nvpair id="scsi-shooter-instance_attributes-pcmk_host_list"
name="pcmk_host_list" value="coro-test-1 coro-test-2"/>
          <nvpair id="scsi-shooter-instance_attributes-pcmk_monitor_action"
name="pcmk_monitor_action" value="metadata"/>
          <nvpair id="scsi-shooter-instance_attributes-pcmk_reboot_action"
name="pcmk_reboot_action" value="off"/>
          <nvpair id="scsi-shooter-instance_attributes-pcmk_delay_max"
name="pcmk_delay_max" value="15"/>
          <nvpair id="scsi-shooter-instance_attributes-devices"
name="devices" value="/dev/sdb1"/>
        </instance_attributes>
        <meta_attributes id="scsi-shooter-meta_attributes">
          <nvpair id="scsi-shooter-meta_attributes-provides"
name="provides" value="unfencing"/>
        </meta_attributes>
        <operations>
          <op id="scsi-shooter-monitor-interval-60s" interval="60s"
name="monitor"/>
        </operations>
      </primitive>
    </resources>
    <constraints>
      <rsc_order first="dlm-clone" first-action="start"
id="order-dlm-clone-clvmd-clone-mandatory" then="clvmd-clone"
then-action="start"/>
      <rsc_colocation id="colocation-clvmd-clone-dlm-clone-INFINITY"
rsc="clvmd-clone" score="INFINITY" with-rsc="dlm-clone"/>
    </constraints>
  </configuration>
</cib>

***
******   -- Console Output -- Comments begin with "#"
***

#
##
###  -- Moments before testing starts (Node 2)--
##
#
root at coro-test-2:~# date; pcs status
Thu Sep  6 08:54:14 EDT 2018
Cluster name: my_first_cluster
Last updated: Thu Sep  6 08:54:14 2018 Last change: Wed Sep  5 16:28:53
2018 by root via cibadmin on coro-test-2
Stack: corosync
Current DC: coro-test-1 (version 1.1.14-70404b0) - partition with quorum
2 nodes and 5 resources configured

Online: [ coro-test-1 coro-test-2 ]

Full list of resources:

 Clone Set: dlm-clone [dlm]
     Started: [ coro-test-1 coro-test-2 ]
 Clone Set: clvmd-clone [clvmd]
     Started: [ coro-test-1 coro-test-2 ]
 scsi-shooter (stonith:fence_scsi): Started coro-test-1

PCSD Status:
  coro-test-1: Online
  coro-test-2: Online

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled
root at coro-test-2:~# /usr/bin/sg_persist -n -i -k -d /dev/sdb1
  PR generation=0x41, 2 registered reservation keys follow:
    0x3ff20000
    0x3ff20001
root at coro-test-2:~# dlm_tool status
cluster nodeid 2 quorate 1 ring seq 496 496
daemon now 58996 fence_pid 0
node 1 M add 92 rem 0 fail 0 fence 0 at 0 0
node 2 M add 92 rem 0 fail 0 fence 0 at 0 0
root at coro-test-2:~# dlm_tool ls
dlm lockspaces
name          clvmd
id            0x4104eefa
flags         0x00000000
change        member 2 joined 1 remove 0 failed 0 seq 1,1
members       1 2


#
##
###  -- Node 1 is fenced --
##
#
root at coro-test-2:~# date; pcs stonith fence coro-test-1
Thu Sep  6 08:55:44 EDT 2018
Node: coro-test-1 fenced
root at coro-test-2:~# pcs status
Cluster name: my_first_cluster
Last updated: Thu Sep  6 08:56:40 2018 Last change: Wed Sep  5 16:28:53
2018 by root via cibadmin on coro-test-2
Stack: corosync
Current DC: coro-test-2 (version 1.1.14-70404b0) - partition with quorum
2 nodes and 5 resources configured

Node coro-test-1: pending
Online: [ coro-test-2 ]

Full list of resources:

 Clone Set: dlm-clone [dlm]
     Started: [ coro-test-2 ]
     Stopped: [ coro-test-1 ]
 Clone Set: clvmd-clone [clvmd]
     Started: [ coro-test-2 ]
     Stopped: [ coro-test-1 ]
 scsi-shooter (stonith:fence_scsi): Stopped

PCSD Status:
  coro-test-1: Online
  coro-test-2: Online

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled
root at coro-test-2:~# /usr/bin/sg_persist -n -i -k -d /dev/sdb1
  PR generation=0x42, 1 registered reservation key follows:
    0x3ff20001
root at coro-test-2:~# dlm_tool ls
dlm lockspaces
name          clvmd
id            0x4104eefa
flags         0x00000000
change        member 2 joined 1 remove 0 failed 0 seq 1,1
members       1 2

root at coro-test-2:~# dlm_tool status
cluster nodeid 2 quorate 1 ring seq 496 496
daemon now 59166 fence_pid 0
node 1 M add 92 rem 0 fail 0 fence 0 at 0 0
node 2 M add 92 rem 0 fail 0 fence 0 at 0 0
root at coro-test-2:~#

#
## NOTE:  DLM still running on other node, coro-test-1, at this point.
#
root at coro-test-1:~# ps -aef | grep dlm
root      1633     1  0 Sep05 ?        00:00:00 dlm_controld -s 0
root      1752     2  0 Sep05 ?        00:00:00 [dlm_scand]
root      1754     2  0 Sep05 ?        00:00:00 [dlm_recv]
root      1755     2  0 Sep05 ?        00:00:00 [dlm_send]
root      1756     2  0 Sep05 ?        00:00:00 [dlm_recoverd]
root     29673  1850  0 08:58 pts/0    00:00:00 grep --color=auto dlm
root at coro-test-1:~#


#
##
###  -- node 2 output after node 1 is shutdown
##
#
#### NODE 1 SHUTDOWN ####
root at coro-test-1:~# pcs status; date; init 0
Error: cluster is not currently running on this node
Thu Sep  6 09:03:38 EDT 2018

#### NODE 2 UI OUTPUT ####
root at coro-test-2:~# pcs status
Cluster name: my_first_cluster
Last updated: Thu Sep  6 09:04:16 2018 Last change: Wed Sep  5 16:28:53
2018 by root via cibadmin on coro-test-2
Stack: corosync
Current DC: coro-test-2 (version 1.1.14-70404b0) - partition with quorum
2 nodes and 5 resources configured

Node coro-test-2: UNCLEAN (online)
OFFLINE: [ coro-test-1 ]

Full list of resources:

 Clone Set: dlm-clone [dlm]
     dlm (ocf::pacemaker:controld): FAILED coro-test-2
     Stopped: [ coro-test-1 ]
 Clone Set: clvmd-clone [clvmd]
     Started: [ coro-test-2 ]
     Stopped: [ coro-test-1 ]
 scsi-shooter (stonith:fence_scsi): Started coro-test-2

Failed Actions:
* dlm_monitor_30000 on coro-test-2 'unknown error' (1): call=17,
status=complete, exitreason='none',
    last-rc-change='Thu Sep  6 09:04:09 2018', queued=0ms, exec=0ms


PCSD Status:
  coro-test-1: Offline
  coro-test-2: Online

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled
root at coro-test-2:~# /usr/bin/sg_persist -n -i -k -d /dev/sdb1
  PR generation=0x42, 1 registered reservation key follows:
    0x3ff20001
root at coro-test-2:~# dlm_tool status
cannot connect to dlm_controld
root at coro-test-2:~# dlm_tool ls
root at coro-test-2:~#

***
******   -- Syslog Output -- Comments begin with "#"
***
#
###
#####  NODE 2 FENCES NODE 1 (pcs stonith fence coro-test-1)
###
#

Sep  6 08:55:55 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[27468]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-1' ]
Sep  6 08:55:55 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[27468]
stderr: [  ]
Sep  6 08:55:55 coro-test-2 stonith-ng[1541]:   notice: Operation 'reboot'
[27468] (call 2 from stonith_admin.27448) for host 'coro-test-1' with
device 'scsi-shooter' returned: 0 (OK)
Sep  6 08:55:55 coro-test-2 stonith-ng[1541]:   notice: Operation reboot of
coro-test-1 by coro-test-2 for stonith_admin.27448 at coro-test-2.5886f6b5: OK
Sep  6 08:55:55 coro-test-2 stonith-ng[1541]:  warning:
new_event_notification (1541-27448-12): Broken pipe (32)
Sep  6 08:55:55 coro-test-2 crmd[1545]:   notice: State transition S_NOT_DC
-> S_ELECTION [ input=I_ELECTION cause=C_CRMD_STATUS_CALLBACK
origin=peer_update_callback ]
Sep  6 08:55:55 coro-test-2 crmd[1545]:   notice: State transition
S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_TIMER_POPPED
origin=election_timeout_popped ]
Sep  6 08:55:55 coro-test-2 attrd[1543]:   notice: crm_update_peer_proc:
Node coro-test-1[1] - state is now lost (was member)
Sep  6 08:55:55 coro-test-2 attrd[1543]:   notice: Removing coro-test-1/1
from the membership list
Sep  6 08:55:55 coro-test-2 attrd[1543]:   notice: Purged 1 peers with id=1
and/or uname=coro-test-1 from the membership cache
Sep  6 08:55:55 coro-test-2 stonith-ng[1541]:   notice:
crm_update_peer_proc: Node coro-test-1[1] - state is now lost (was member)
Sep  6 08:55:55 coro-test-2 stonith-ng[1541]:   notice: Removing
coro-test-1/1 from the membership list
Sep  6 08:55:55 coro-test-2 stonith-ng[1541]:   notice: Purged 1 peers with
id=1 and/or uname=coro-test-1 from the membership cache
Sep  6 08:55:55 coro-test-2 cib[1540]:   notice: crm_update_peer_proc: Node
coro-test-1[1] - state is now lost (was member)
Sep  6 08:55:55 coro-test-2 cib[1540]:   notice: Removing coro-test-1/1
from the membership list
Sep  6 08:55:55 coro-test-2 cib[1540]:   notice: Purged 1 peers with id=1
and/or uname=coro-test-1 from the membership cache
Sep  6 08:55:55 coro-test-2 crmd[1545]:  warning: FSA: Input I_ELECTION_DC
from do_election_check() received in state S_INTEGRATION
Sep  6 08:55:55 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 08:55:55 coro-test-2 pengine[1544]:  warning: Action dlm:0_monitor_0
on coro-test-1 is unrunnable (pending)
Sep  6 08:55:55 coro-test-2 pengine[1544]:  warning: Action
clvmd:0_monitor_0 on coro-test-1 is unrunnable (pending)
Sep  6 08:55:55 coro-test-2 pengine[1544]:  warning: Action
scsi-shooter_monitor_0 on coro-test-1 is unrunnable (pending)
Sep  6 08:55:55 coro-test-2 pengine[1544]:   notice: Unfencing coro-test-1:
node discovery
Sep  6 08:55:55 coro-test-2 pengine[1544]:   notice: Start
 scsi-shooter#011(coro-test-2 - blocked)
Sep  6 08:55:55 coro-test-2 stonith-ng[1541]:    error: Operation on of
coro-test-1 by <no-one> for crmd.1545 at coro-test-2.b06d9d39: No such device
Sep  6 08:55:55 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 08:55:55 coro-test-2 crmd[1545]:   notice: Transition 0 (Complete=1,
Pending=0, Fired=0, Skipped=0, Incomplete=2,
Source=/var/lib/pacemaker/pengine/pe-input-107.bz2): Complete
Sep  6 08:55:55 coro-test-2 crmd[1545]:   notice: State transition
S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL
origin=notify_crmd ]
Sep  6 08:55:55 coro-test-2 pengine[1544]:   notice: Calculated Transition
0: /var/lib/pacemaker/pengine/pe-input-107.bz2
Sep  6 08:58:22 coro-test-2 dhclient[1070]: DHCPREQUEST of 10.0.2.5 on
enp0s3 to 10.0.2.3 port 67 (xid=0x1c247c88)
Sep  6 08:58:22 coro-test-2 dhclient[1070]: DHCPACK of 10.0.2.5 from
10.0.2.3
Sep  6 08:58:22 coro-test-2 dhclient[1070]: bound to 10.0.2.5 -- renewal in
466 seconds.
Sep  6 08:59:20 coro-test-2 dhclient[860]: DHCPREQUEST of 192.168.57.4 on
enp0s8 to 192.168.57.2 port 67 (xid=0x5dfb57d7)
Sep  6 08:59:20 coro-test-2 dhclient[860]: DHCPACK of 192.168.57.4 from
192.168.57.2
Sep  6 08:59:20 coro-test-2 dhclient[860]: bound to 192.168.57.4 -- renewal
in 599 seconds.

#
###
#####   NODE 1 IS SHUTDOWN
###
#

##
###     Cluster appears to see the loss of node 1
##
Sep  6 09:03:38 coro-test-2 corosync[1505]: notice  [TOTEM ] A new
membership (10.0.2.5:500) was formed. Members left: 1
Sep  6 09:03:38 coro-test-2 corosync[1505]:  [TOTEM ] A new membership (
10.0.2.5:500) was formed. Members left: 1
Sep  6 09:03:38 coro-test-2 pacemakerd[1537]:   notice:
crm_reap_unseen_nodes: Node coro-test-1[1] - state is now lost (was member)
Sep  6 09:03:38 coro-test-2 dlm_controld[1657]: 59550 fence request 1 pid
28247 procdown time 1536239018 fence_all dlm_stonith
Sep  6 09:03:38 coro-test-2 crmd[1545]:   notice: crm_reap_unseen_nodes:
Node coro-test-1[1] - state is now lost (was member)
Sep  6 09:03:38 coro-test-2 dlm_controld[1657]: 59550 tell corosync to
remove nodeid 1 from cluster

##
###     DLM appears to close off connection to NODE 1
##
Sep  6 09:03:38 coro-test-2 kernel: [59552.844406] dlm: closing connection
to node 1
Sep  6 09:03:38 coro-test-2 dlm_stonith: stonith_api_time: Found 2 entries
for 1/(null): 0 in progress, 1 completed
Sep  6 09:03:38 coro-test-2 dlm_stonith: stonith_api_time: Node 1/(null)
last kicked at: 1536238555

##
###     Stonith attempts to fence node 1 (again; it was already fenced)
##
Sep  6 09:03:42 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28248]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-1' ]
Sep  6 09:03:42 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28248]
stderr: [  ]
Sep  6 09:03:42 coro-test-2 stonith-ng[1541]:   notice: Operation 'reboot'
[28248] (call 2 from stonith-api.28247) for host 'coro-test-1' with device
'scsi-shooter' returned: 0 (OK)
Sep  6 09:03:42 coro-test-2 stonith-ng[1541]:   notice: Operation reboot of
coro-test-1 by coro-test-2 for stonith-api.28247 at coro-test-2.2365d094: OK
Sep  6 09:03:42 coro-test-2 stonith-api[28247]: stonith_api_kick: Node
1/(null) kicked: reboot
Sep  6 09:03:42 coro-test-2 crmd[1545]:   notice: State transition S_IDLE
-> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Sep  6 09:03:42 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore

##
###     I begin to get confused here:  Why is dlm closing the connection to
the node 2 (itself??)
##
Sep  6 09:03:42 coro-test-2 kernel: [59556.684699] dlm: closing connection
to node 2

##
###     LOCKSPACE GOES ABANDONED!!  why?  WHY????
##
Sep  6 09:03:42 coro-test-2 dlm_controld[1657]: 59554 abandoned lockspace
clvmd

##
###     Alas, my cluster begins to topple itself
##
Sep  6 09:03:42 coro-test-2 pengine[1544]:   notice: Start
 scsi-shooter#011(coro-test-2)
Sep  6 09:03:42 coro-test-2 pengine[1544]:   notice: Calculated Transition
1: /var/lib/pacemaker/pengine/pe-input-108.bz2
Sep  6 09:03:42 coro-test-2 kernel: [59556.693578] dlm: dlm user daemon
left 1 lockspaces
Sep  6 09:03:43 coro-test-2 crmd[1545]:   notice: Operation
scsi-shooter_start_0: ok (node=coro-test-2, call=20, rc=0, cib-update=56,
confirmed=true)
Sep  6 09:03:43 coro-test-2 crmd[1545]:   notice: Transition 1 (Complete=2,
Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-108.bz2): Complete
Sep  6 09:03:43 coro-test-2 crmd[1545]:   notice: State transition
S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL
origin=notify_crmd ]
Sep  6 09:04:05 coro-test-2 controld(dlm)[28324]: ERROR: Uncontrolled
lockspace exists, system must reboot. Executing suicide fencing
Sep  6 09:04:07 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:07 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:08 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28344]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:08 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28344]
stderr: [  ]
Sep  6 09:04:08 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28344]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:08 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28344]
stderr: [  ]
Sep  6 09:04:08 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28344]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:08 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28344]
stderr: [  ]
Sep  6 09:04:08 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28344]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:08 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28344]
stderr: [  ]
Sep  6 09:04:08 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28344]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:08 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28344]
stderr: [  ]
Sep  6 09:04:09 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:09 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28357]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28357]
stderr: [  ]
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28357]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28357]
stderr: [  ]
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28357]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28357]
stderr: [  ]
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28357]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28357]
stderr: [  ]
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28357]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28357]
stderr: [  ]
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:    error: Operation 'reboot'
[28357] (call 2 from stonith_admin.controld.28343) for host 'coro-test-2'
with device 'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:04:09 coro-test-2 stonith-ng[1541]:    error: Operation reboot of
coro-test-2 by coro-test-2 for
stonith_admin.controld.28343 at coro-test-2.fd726e0b: Generic Pacemaker error
Sep  6 09:04:09 coro-test-2 crmd[1545]:   notice:
coro-test-2-dlm_monitor_30000:17 [ Command failed: Generic Pacemaker
error\n ]
Sep  6 09:04:09 coro-test-2 crmd[1545]:   notice: State transition S_IDLE
-> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:04:09 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:04:09 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:04:09 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:04:09 coro-test-2 pengine[1544]:  warning: Calculated Transition
2: /var/lib/pacemaker/pengine/pe-warn-68.bz2
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:04:09 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:04:09 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:04:09 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:04:09 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:04:09 coro-test-2 pengine[1544]:  warning: Calculated Transition
3: /var/lib/pacemaker/pengine/pe-warn-69.bz2
Sep  6 09:04:13 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:13 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:13 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28370]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:13 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28370]
stderr: [  ]
Sep  6 09:04:13 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28370]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:13 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28370]
stderr: [  ]
Sep  6 09:04:13 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28370]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:13 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28370]
stderr: [  ]
Sep  6 09:04:13 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28370]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:13 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28370]
stderr: [  ]
Sep  6 09:04:13 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28370]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:13 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28370]
stderr: [  ]
Sep  6 09:04:14 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:14 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28383]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28383]
stderr: [  ]
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28383]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28383]
stderr: [  ]
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28383]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28383]
stderr: [  ]
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28383]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28383]
stderr: [  ]
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28383]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28383]
stderr: [  ]
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:    error: Operation 'off'
[28383] (call 3 from crmd.1545) for host 'coro-test-2' with device
'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:04:14 coro-test-2 stonith-ng[1541]:    error: Operation off of
coro-test-2 by coro-test-2 for crmd.1545 at coro-test-2.171aafb6: Generic
Pacemaker error
Sep  6 09:04:14 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 09:04:14 coro-test-2 crmd[1545]:   notice: Transition 3 (Complete=1,
Pending=0, Fired=0, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-warn-69.bz2): Complete
Sep  6 09:04:14 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:04:14 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:04:14 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:04:14 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:04:14 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:04:14 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:04:14 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:04:14 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:04:14 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:04:14 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:04:14 coro-test-2 pengine[1544]:  warning: Calculated Transition
4: /var/lib/pacemaker/pengine/pe-warn-69.bz2
Sep  6 09:04:19 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:19 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:19 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28432]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:19 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28432]
stderr: [  ]
Sep  6 09:04:19 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28432]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:19 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28432]
stderr: [  ]
Sep  6 09:04:19 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28432]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:19 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28432]
stderr: [  ]
Sep  6 09:04:19 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28432]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:19 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28432]
stderr: [  ]
Sep  6 09:04:19 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28432]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:19 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28432]
stderr: [  ]
Sep  6 09:04:20 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:20 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28445]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28445]
stderr: [  ]
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28445]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28445]
stderr: [  ]
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28445]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28445]
stderr: [  ]
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28445]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28445]
stderr: [  ]
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28445]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28445]
stderr: [  ]
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:    error: Operation 'off'
[28445] (call 4 from crmd.1545) for host 'coro-test-2' with device
'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:04:20 coro-test-2 stonith-ng[1541]:    error: Operation off of
coro-test-2 by coro-test-2 for crmd.1545 at coro-test-2.0b907f76: Generic
Pacemaker error
Sep  6 09:04:20 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 09:04:20 coro-test-2 crmd[1545]:   notice: Transition 4 (Complete=1,
Pending=0, Fired=0, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-warn-69.bz2): Complete
Sep  6 09:04:20 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:04:20 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:04:20 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:04:20 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:04:20 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:04:20 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:04:20 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:04:20 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:04:20 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:04:20 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:04:20 coro-test-2 pengine[1544]:  warning: Calculated Transition
5: /var/lib/pacemaker/pengine/pe-warn-69.bz2
Sep  6 09:04:24 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:24 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:24 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28477]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:24 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28477]
stderr: [  ]
Sep  6 09:04:24 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28477]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:24 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28477]
stderr: [  ]
Sep  6 09:04:24 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28477]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:24 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28477]
stderr: [  ]
Sep  6 09:04:24 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28477]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:24 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28477]
stderr: [  ]
Sep  6 09:04:24 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28477]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:24 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28477]
stderr: [  ]
Sep  6 09:04:25 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:25 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28490]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28490]
stderr: [  ]
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28490]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28490]
stderr: [  ]
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28490]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28490]
stderr: [  ]
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28490]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28490]
stderr: [  ]
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28490]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28490]
stderr: [  ]
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:    error: Operation 'off'
[28490] (call 5 from crmd.1545) for host 'coro-test-2' with device
'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:04:25 coro-test-2 stonith-ng[1541]:    error: Operation off of
coro-test-2 by coro-test-2 for crmd.1545 at coro-test-2.559102cb: Generic
Pacemaker error
Sep  6 09:04:25 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 09:04:25 coro-test-2 crmd[1545]:   notice: Transition 5 (Complete=1,
Pending=0, Fired=0, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-warn-69.bz2): Complete
Sep  6 09:04:25 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:04:25 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:04:25 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:04:25 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:04:25 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:04:25 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:04:25 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:04:25 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:04:25 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:04:25 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:04:25 coro-test-2 pengine[1544]:  warning: Calculated Transition
6: /var/lib/pacemaker/pengine/pe-warn-69.bz2
Sep  6 09:04:39 coro-test-2 controld(dlm)[28520]: ERROR: Uncontrolled
lockspace exists, system must reboot. Executing suicide fencing
Sep  6 09:04:39 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:39 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:39 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28540]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:39 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28540]
stderr: [  ]
Sep  6 09:04:39 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28540]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:39 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28540]
stderr: [  ]
Sep  6 09:04:39 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28540]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:39 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28540]
stderr: [  ]
Sep  6 09:04:39 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28540]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:39 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28540]
stderr: [  ]
Sep  6 09:04:39 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28540]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:39 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28540]
stderr: [  ]
Sep  6 09:04:40 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:40 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28553]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28553]
stderr: [  ]
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28553]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28553]
stderr: [  ]
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28553]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28553]
stderr: [  ]
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28553]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28553]
stderr: [  ]
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28553]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28553]
stderr: [  ]
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:    error: Operation 'off'
[28553] (call 6 from crmd.1545) for host 'coro-test-2' with device
'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:04:40 coro-test-2 stonith-ng[1541]:    error: Operation off of
coro-test-2 by coro-test-2 for crmd.1545 at coro-test-2.58604e64: Generic
Pacemaker error
Sep  6 09:04:40 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 09:04:40 coro-test-2 crmd[1545]:   notice: Transition 6 (Complete=1,
Pending=0, Fired=0, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-warn-69.bz2): Complete
Sep  6 09:04:40 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:04:40 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:04:40 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:04:40 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:04:40 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:04:40 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:04:40 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:04:40 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:04:40 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:04:40 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:04:40 coro-test-2 pengine[1544]:  warning: Calculated Transition
7: /var/lib/pacemaker/pengine/pe-warn-69.bz2
Sep  6 09:04:49 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:49 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:49 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28568]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:49 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28568]
stderr: [  ]
Sep  6 09:04:49 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28568]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:49 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28568]
stderr: [  ]
Sep  6 09:04:49 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28568]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:49 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28568]
stderr: [  ]
Sep  6 09:04:49 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28568]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:49 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28568]
stderr: [  ]
Sep  6 09:04:49 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28568]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:49 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28568]
stderr: [  ]
Sep  6 09:04:50 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:50 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28581]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28581]
stderr: [  ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28581]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28581]
stderr: [  ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28581]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28581]
stderr: [  ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28581]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28581]
stderr: [  ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28581]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28581]
stderr: [  ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:    error: Operation 'reboot'
[28581] (call 2 from stonith_admin.controld.28539) for host 'coro-test-2'
with device 'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:    error: Operation reboot of
coro-test-2 by coro-test-2 for
stonith_admin.controld.28539 at coro-test-2.29f17c2f: Generic Pacemaker error
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning:
new_event_notification (1541-28539-12): Broken pipe (32)
Sep  6 09:04:51 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:51 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28594]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28594]
stderr: [  ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28594]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28594]
stderr: [  ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28594]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28594]
stderr: [  ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28594]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28594]
stderr: [  ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28594]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28594]
stderr: [  ]
Sep  6 09:04:52 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:04:52 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28607]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28607]
stderr: [  ]
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28607]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28607]
stderr: [  ]
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28607]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28607]
stderr: [  ]
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28607]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28607]
stderr: [  ]
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28607]
stderr: [ Please use '-h' for usage ]
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28607]
stderr: [  ]
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:    error: Operation 'off'
[28607] (call 7 from crmd.1545) for host 'coro-test-2' with device
'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:04:52 coro-test-2 stonith-ng[1541]:    error: Operation off of
coro-test-2 by coro-test-2 for crmd.1545 at coro-test-2.1c099fde: Generic
Pacemaker error
Sep  6 09:04:52 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 09:04:52 coro-test-2 crmd[1545]:   notice: Transition 7 (Complete=1,
Pending=0, Fired=0, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-warn-69.bz2): Complete
Sep  6 09:04:52 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:04:52 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:04:52 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:04:52 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:04:52 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:04:52 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:04:52 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:04:52 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:04:52 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:04:52 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:04:52 coro-test-2 pengine[1544]:  warning: Calculated Transition
8: /var/lib/pacemaker/pengine/pe-warn-69.bz2
Sep  6 09:05:01 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:01 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:01 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28643]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:01 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28643]
stderr: [  ]
Sep  6 09:05:01 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28643]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:01 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28643]
stderr: [  ]
Sep  6 09:05:01 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28643]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:01 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28643]
stderr: [  ]
Sep  6 09:05:01 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28643]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:01 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28643]
stderr: [  ]
Sep  6 09:05:01 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28643]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:01 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28643]
stderr: [  ]
Sep  6 09:05:02 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:02 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28656]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28656]
stderr: [  ]
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28656]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28656]
stderr: [  ]
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28656]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28656]
stderr: [  ]
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28656]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28656]
stderr: [  ]
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28656]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28656]
stderr: [  ]
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:    error: Operation 'off'
[28656] (call 8 from crmd.1545) for host 'coro-test-2' with device
'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:05:02 coro-test-2 stonith-ng[1541]:    error: Operation off of
coro-test-2 by coro-test-2 for crmd.1545 at coro-test-2.41c7a339: Generic
Pacemaker error
Sep  6 09:05:02 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 09:05:02 coro-test-2 crmd[1545]:   notice: Transition 8 (Complete=1,
Pending=0, Fired=0, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-warn-69.bz2): Complete
Sep  6 09:05:02 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:05:02 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:05:02 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:05:02 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:05:02 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:05:02 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:05:02 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:05:02 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:05:02 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:05:02 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:05:02 coro-test-2 pengine[1544]:  warning: Calculated Transition
9: /var/lib/pacemaker/pengine/pe-warn-69.bz2
Sep  6 09:05:16 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:16 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:16 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28669]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:16 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28669]
stderr: [  ]
Sep  6 09:05:16 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28669]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:16 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28669]
stderr: [  ]
Sep  6 09:05:16 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28669]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:16 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28669]
stderr: [  ]
Sep  6 09:05:16 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28669]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:16 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28669]
stderr: [  ]
Sep  6 09:05:16 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28669]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:16 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28669]
stderr: [  ]
Sep  6 09:05:17 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:17 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28682]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28682]
stderr: [  ]
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28682]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28682]
stderr: [  ]
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28682]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28682]
stderr: [  ]
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28682]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28682]
stderr: [  ]
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28682]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28682]
stderr: [  ]
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:    error: Operation 'off'
[28682] (call 9 from crmd.1545) for host 'coro-test-2' with device
'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:05:18 coro-test-2 stonith-ng[1541]:    error: Operation off of
coro-test-2 by coro-test-2 for crmd.1545 at coro-test-2.16fd9d3b: Generic
Pacemaker error
Sep  6 09:05:18 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 09:05:18 coro-test-2 crmd[1545]:   notice: Transition 9 (Complete=1,
Pending=0, Fired=0, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-warn-69.bz2): Complete
Sep  6 09:05:18 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:05:18 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:05:18 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:05:18 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:05:18 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:05:18 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:05:18 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:05:18 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:05:18 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:05:18 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:05:18 coro-test-2 pengine[1544]:  warning: Calculated Transition
10: /var/lib/pacemaker/pengine/pe-warn-69.bz2
Sep  6 09:05:21 coro-test-2 controld(dlm)[28695]: ERROR: Uncontrolled
lockspace exists, system must reboot. Executing suicide fencing
Sep  6 09:05:29 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:29 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:29 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28751]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:29 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28751]
stderr: [  ]
Sep  6 09:05:29 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28751]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:29 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28751]
stderr: [  ]
Sep  6 09:05:29 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28751]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:29 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28751]
stderr: [  ]
Sep  6 09:05:29 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28751]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:29 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28751]
stderr: [  ]
Sep  6 09:05:29 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28751]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:29 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28751]
stderr: [  ]
Sep  6 09:05:30 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:30 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28764]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28764]
stderr: [  ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28764]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28764]
stderr: [  ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28764]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28764]
stderr: [  ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28764]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28764]
stderr: [  ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28764]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28764]
stderr: [  ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:    error: Operation 'off'
[28764] (call 10 from crmd.1545) for host 'coro-test-2' with device
'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:    error: Operation off of
coro-test-2 by coro-test-2 for crmd.1545 at coro-test-2.c6433fd3: Generic
Pacemaker error
Sep  6 09:05:30 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 09:05:30 coro-test-2 crmd[1545]:   notice: Transition 10
(Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-warn-69.bz2): Complete
Sep  6 09:05:30 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:05:30 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:05:30 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:05:30 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:05:30 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:05:30 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:05:30 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:05:30 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:05:30 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:05:30 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:05:30 coro-test-2 pengine[1544]:  warning: Calculated Transition
11: /var/lib/pacemaker/pengine/pe-warn-69.bz2
Sep  6 09:05:30 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:30 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28777]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28777]
stderr: [  ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28777]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28777]
stderr: [  ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28777]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28777]
stderr: [  ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28777]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28777]
stderr: [  ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28777]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:30 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28777]
stderr: [  ]
Sep  6 09:05:31 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:31 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28790]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28790]
stderr: [  ]
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28790]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28790]
stderr: [  ]
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28790]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28790]
stderr: [  ]
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28790]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28790]
stderr: [  ]
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28790]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28790]
stderr: [  ]
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:    error: Operation 'reboot'
[28790] (call 2 from stonith_admin.controld.28714) for host 'coro-test-2'
with device 'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:05:31 coro-test-2 stonith-ng[1541]:    error: Operation reboot of
coro-test-2 by coro-test-2 for
stonith_admin.controld.28714 at coro-test-2.ffe64359: Generic Pacemaker error
Sep  6 09:05:36 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:36 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:36 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28803]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:36 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28803]
stderr: [  ]
Sep  6 09:05:36 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28803]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:36 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28803]
stderr: [  ]
Sep  6 09:05:36 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28803]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:36 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28803]
stderr: [  ]
Sep  6 09:05:36 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28803]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:36 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28803]
stderr: [  ]
Sep  6 09:05:36 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28803]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:36 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28803]
stderr: [  ]
Sep  6 09:05:37 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:37 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28816]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28816]
stderr: [  ]
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28816]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28816]
stderr: [  ]
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28816]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28816]
stderr: [  ]
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28816]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28816]
stderr: [  ]
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28816]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28816]
stderr: [  ]
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:    error: Operation 'off'
[28816] (call 11 from crmd.1545) for host 'coro-test-2' with device
'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:05:37 coro-test-2 stonith-ng[1541]:    error: Operation off of
coro-test-2 by coro-test-2 for crmd.1545 at coro-test-2.7dfe97e8: Generic
Pacemaker error
Sep  6 09:05:37 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 09:05:37 coro-test-2 crmd[1545]:   notice: Transition 11
(Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-warn-69.bz2): Complete
Sep  6 09:05:37 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:05:37 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:05:37 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:05:37 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:05:37 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:05:37 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:05:37 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:05:37 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:05:37 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:05:37 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:05:37 coro-test-2 pengine[1544]:  warning: Calculated Transition
12: /var/lib/pacemaker/pengine/pe-warn-69.bz2
Sep  6 09:05:50 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:50 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:50 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28829]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:50 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28829]
stderr: [  ]
Sep  6 09:05:50 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28829]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:50 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28829]
stderr: [  ]
Sep  6 09:05:50 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28829]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:50 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28829]
stderr: [  ]
Sep  6 09:05:50 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28829]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:50 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28829]
stderr: [  ]
Sep  6 09:05:50 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28829]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:50 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28829]
stderr: [  ]
Sep  6 09:05:51 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:05:51 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28842]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28842]
stderr: [  ]
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28842]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28842]
stderr: [  ]
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28842]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28842]
stderr: [  ]
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28842]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28842]
stderr: [  ]
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28842]
stderr: [ Please use '-h' for usage ]
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28842]
stderr: [  ]
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:    error: Operation 'off'
[28842] (call 12 from crmd.1545) for host 'coro-test-2' with device
'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:05:51 coro-test-2 stonith-ng[1541]:    error: Operation off of
coro-test-2 by coro-test-2 for crmd.1545 at coro-test-2.fb005423: Generic
Pacemaker error
Sep  6 09:05:51 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 09:05:51 coro-test-2 crmd[1545]:   notice: Transition 12
(Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-warn-69.bz2): Complete
Sep  6 09:05:51 coro-test-2 pengine[1544]:   notice: On loss of CCM Quorum:
Ignore
Sep  6 09:05:51 coro-test-2 pengine[1544]:  warning: Processing failed op
monitor for dlm:0 on coro-test-2: unknown error (1)
Sep  6 09:05:51 coro-test-2 pengine[1544]:  warning: Node coro-test-2 will
be fenced because of resource failure(s)
Sep  6 09:05:51 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:0
with instance of dlm-clone
Sep  6 09:05:51 coro-test-2 pengine[1544]:   notice: Cannot pair clvmd:1
with instance of dlm-clone
Sep  6 09:05:51 coro-test-2 pengine[1544]:  warning: Scheduling Node
coro-test-2 for STONITH
Sep  6 09:05:51 coro-test-2 pengine[1544]:   notice: Stop of failed
resource dlm:0 is implicit after coro-test-2 is fenced
Sep  6 09:05:51 coro-test-2 pengine[1544]:   notice: Stop
dlm:0#011(coro-test-2)
Sep  6 09:05:51 coro-test-2 pengine[1544]:   notice: Stop
clvmd:0#011(coro-test-2)
Sep  6 09:05:51 coro-test-2 pengine[1544]:   notice: Stop
scsi-shooter#011(coro-test-2)
Sep  6 09:05:51 coro-test-2 pengine[1544]:  warning: Calculated Transition
13: /var/lib/pacemaker/pengine/pe-warn-69.bz2
Sep  6 09:06:01 coro-test-2 controld(dlm)[28874]: ERROR: Uncontrolled
lockspace exists, system must reboot. Executing suicide fencing
Sep  6 09:06:02 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:06:02 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:06:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28894]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:06:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28894]
stderr: [  ]
Sep  6 09:06:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28894]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:06:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28894]
stderr: [  ]
Sep  6 09:06:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28894]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:06:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28894]
stderr: [  ]
Sep  6 09:06:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28894]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:06:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28894]
stderr: [  ]
Sep  6 09:06:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28894]
stderr: [ Please use '-h' for usage ]
Sep  6 09:06:02 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28894]
stderr: [  ]
Sep  6 09:06:03 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:06:03 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28907]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28907]
stderr: [  ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28907]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28907]
stderr: [  ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28907]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28907]
stderr: [  ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28907]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28907]
stderr: [  ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28907]
stderr: [ Please use '-h' for usage ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28907]
stderr: [  ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:    error: Operation 'off'
[28907] (call 13 from crmd.1545) for host 'coro-test-2' with device
'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:    error: Operation off of
coro-test-2 by coro-test-2 for crmd.1545 at coro-test-2.e8cc12be: Generic
Pacemaker error
Sep  6 09:06:03 coro-test-2 crmd[1545]:   notice: Transition aborted:
Stonith failed (source=tengine_stonith_callback:748, 0)
Sep  6 09:06:03 coro-test-2 crmd[1545]:   notice: Transition 13
(Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=9,
Source=/var/lib/pacemaker/pengine/pe-warn-69.bz2): Complete
Sep  6 09:06:03 coro-test-2 crmd[1545]:   notice: State transition
S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL
origin=notify_crmd ]
Sep  6 09:06:03 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:06:03 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28922]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28922]
stderr: [  ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28922]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28922]
stderr: [  ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28922]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28922]
stderr: [  ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28922]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28922]
stderr: [  ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28922]
stderr: [ Please use '-h' for usage ]
Sep  6 09:06:03 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28922]
stderr: [  ]
Sep  6 09:06:04 coro-test-2 fence_scsi: Failed: keys cannot be same. You
can not fence yourself.
Sep  6 09:06:04 coro-test-2 fence_scsi: Please use '-h' for usage
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28935]
stderr: [ WARNING:root:Parse error: Ignoring unknown option
'port=coro-test-2' ]
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28935]
stderr: [  ]
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28935]
stderr: [ ERROR:root:Failed: keys cannot be same. You can not fence
yourself. ]
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28935]
stderr: [  ]
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28935]
stderr: [ Failed: keys cannot be same. You can not fence yourself. ]
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28935]
stderr: [  ]
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28935]
stderr: [ ERROR:root:Please use '-h' for usage ]
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28935]
stderr: [  ]
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28935]
stderr: [ Please use '-h' for usage ]
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:  warning: fence_scsi[28935]
stderr: [  ]
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:    error: Operation 'reboot'
[28935] (call 2 from stonith_admin.controld.28893) for host 'coro-test-2'
with device 'scsi-shooter' returned: -201 (Generic Pacemaker error)
Sep  6 09:06:04 coro-test-2 stonith-ng[1541]:    error: Operation reboot of
coro-test-2 by coro-test-2 for
stonith_admin.controld.28893 at coro-test-2.185da33b: Generic Pacemaker error
Sep  6 09:06:08 coro-test-2 dhclient[1070]: DHCPREQUEST of 10.0.2.5 on
enp0s3 to 10.0.2.3 port 67 (xid=0x1c247c88)
Sep  6 09:06:08 coro-test-2 dhclient[1070]: DHCPACK of 10.0.2.5 from
10.0.2.3
Sep  6 09:06:08 coro-test-2 dhclient[1070]: bound to 10.0.2.5 -- renewal in
571 seconds.


-- 
Patrick Whitney
DevOps Engineer -- Tools
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20180906/7c7b926c/attachment-0001.html>


More information about the Users mailing list