<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=windows-1251">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hi!<br>
</p>
<div class="moz-forward-container"><br>
In continuation of prevoius mails, now I have more complex setup.
Our hardware are capable of two STONITH methods: ILO and SCSI
persistent reservations on shared storage. First method works
fine, nevertheless, sometimes in the past we faced problems with
inaccessible ILO devices or something... So, we would like to have
SCSI fencing as an additional method.<br>
<br>
The problem: when a node 2 recovers, some resources are just
stopped and restarted on node 1. As far as I understand, primitive
resources are affected, but clone instances are not affected.<br>
<br>
In the example below, when bvnode2 recovers, vm_smartbv1 is
restarted on bvnode1, and vm_smartbv2 is live-migrated without
interruption to bvnode2. All other resources are clones working on
bvnode1 and they are unaffected.<br>
<br>
If I set "meta requires=fencing" for vm resources, they are not
restarted anymore. But why unfencing of bvnode2 affects resources
running on bvnode1?<br>
<br>
<br>
====<br>
<p class="MsoNormal"><span lang="EN-US">Current cluster status:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Online: [ bvnode1 bvnode2
]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">ilo.bvnode2
(stonith:fence_ilo4): Started bvnode1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">ilo.bvnode1
(stonith:fence_ilo4): Stopped<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Clone Set: dlm-clone [dlm]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Started: [ bvnode1 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Stopped: [ bvnode2 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Clone Set: clvmd-clone
[clvmd]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Started: [ bvnode1 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Stopped: [ bvnode2 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Clone Set:
cluster-config-clone [cluster-config]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Started: [ bvnode1 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Stopped: [ bvnode2 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">vm_smartbv1
(ocf::heartbeat:VirtualDomain): Started bvnode1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">vm_smartbv2
(ocf::heartbeat:VirtualDomain): Started bvnode1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Clone Set: libvirtd-clone
[libvirtd]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Started: [ bvnode1 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Stopped: [ bvnode2 ]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">storage.bvnode1
(stonith:fence_mpath): Started bvnode1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">storage.bvnode2
(stonith:fence_mpath): Started bvnode1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Transition Summary:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">* Start ilo.bvnode1
(bvnode2)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">* Start dlm:1
(bvnode2)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">* Start clvmd:1
(bvnode2)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">* Start
cluster-config:1 (bvnode2)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">* Restart vm_smartbv1
(Started bvnode1)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">* Migrate vm_smartbv2
(Started bvnode1 -> bvnode2)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"></span>* Start
libvirtd:1 (bvnode2)<o:p></o:p></p>
<span lang="EN-US">* Move storage.bvnode2 (Started bvnode1
-> bvnode2)<br>
====<br>
<br>
</span>Cluster config:<br>
<br>
====<br>
<span lang="EN-US">Cluster Name: smartbvcluster<o:p></o:p></span>
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Corosync Nodes:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">bvnode1 bvnode2<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Pacemaker Nodes:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">bvnode1 bvnode2<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Resources:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Clone: dlm-clone<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Meta Attrs:
interleave=true ordered=true<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Resource: dlm
(class=ocf provider=pacemaker type=controld)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Operations: start
interval=0s timeout=90 (dlm-start-interval-0s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> stop
interval=0s timeout=100 (dlm-stop-interval-0s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> monitor
interval=30s (dlm-monitor-interval-30s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Clone: clvmd-clone<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Meta Attrs:
interleave=true ordered=true<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Resource: clvmd
(class=ocf provider=heartbeat type=clvm)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Operations: start
interval=0s timeout=90 (clvmd-start-interval-0s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> stop
interval=0s timeout=90 (clvmd-stop-interval-0s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> monitor
interval=30s (clvmd-monitor-interval-30s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Clone:
cluster-config-clone<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Meta Attrs:
interleave=true<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Resource:
cluster-config (class=ocf provider=heartbeat
type=Filesystem)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Attributes:
device=/dev/vg_bv_shared/cluster-config
directory=/opt/cluster-config fstype=gfs2 options=noatime<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Operations: start
interval=0s timeout=60 (cluster-config-start-interval-0s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> stop
interval=0s timeout=60 (cluster-config-stop-interval-0s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> monitor
interval=10s on-fail=fence OCF_CHECK_LEVEL=20
(cluster-config-monitor-interval-10s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Resource: vm_smartbv1
(class=ocf provider=heartbeat type=VirtualDomain)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Attributes:
config=/opt/cluster-config/libvirt/qemu/smartbv1.xml
hypervisor=qemu:///system migration_transport=tcp<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Meta Attrs:
allow-migrate=true<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Operations: start
interval=0s timeout=90 (vm_smartbv1-start-interval-0s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> stop
interval=0s timeout=90 (vm_smartbv1-stop-interval-0s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> monitor
interval=10 timeout=30 (vm_smartbv1-monitor-interval-10)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Resource: vm_smartbv2
(class=ocf provider=heartbeat type=VirtualDomain)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Attributes:
config=/opt/cluster-config/libvirt/qemu/smartbv2.xml
hypervisor=qemu:///system migration_transport=tcp<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Meta Attrs:
target-role=started allow-migrate=true<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Operations: start
interval=0s timeout=90 (vm_smartbv2-start-interval-0s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> stop
interval=0s timeout=90 (vm_smartbv2-stop-interval-0s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> monitor
interval=10 timeout=30 (vm_smartbv2-monitor-interval-10)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Clone: libvirtd-clone<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Meta Attrs:
interleave=true<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Resource: libvirtd
(class=systemd type=libvirtd)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Operations: monitor
interval=60s (libvirtd-monitor-interval-60s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Stonith Devices:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Resource: ilo.bvnode2
(class=stonith type=fence_ilo4)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Attributes:
ipaddr=ilo.bvnode2 login=hacluster passwd=s
pcmk_host_list=bvnode2 privlvl=operator<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Operations: monitor
interval=60s (ilo.bvnode2-monitor-interval-60s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Resource: ilo.bvnode1
(class=stonith type=fence_ilo4)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Attributes:
ipaddr=ilo.bvnode1 login=hacluster passwd=s
pcmk_host_list=bvnode1 privlvl=operator<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Operations: monitor
interval=60s (ilo.bvnode1-monitor-interval-60s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Resource:
storage.bvnode1 (class=stonith type=fence_mpath)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Attributes:
key=ab2ee06 pcmk_reboot_action=off
devices=/dev/mapper/mpatha pcmk_host_check=static-list
pcmk_host_list=bvnode1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Meta Attrs:
provides=unfencing<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Operations: monitor
interval=60s (storage.bvnode1-monitor-interval-60s)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Resource:
storage.bvnode2 (class=stonith type=fence_mpath)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Attributes:
key=ab2ee07 pcmk_reboot_action=off
devices=/dev/mapper/mpatha pcmk_host_check=static-list
pcmk_host_list=bvnode2<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Meta Attrs:
provides=unfencing<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Operations: monitor
interval=60s (storage.bvnode2-monitor-interval-60s)<o:p></o:p></span></p>
<p class="MsoNormal">Fencing Levels:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Node: bvnode1<o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> Level 10 - ilo.bvnode1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Node: bvnode2<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Level 10 - ilo.bvnode2<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Location Constraints:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Resource: ilo.bvnode1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Disabled on: bvnode1
(score:-INFINITY)
(id:location-ilo.bvnode1-bvnode1--INFINITY)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Resource: ilo.bvnode2<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> Disabled on: bvnode2
(score:-INFINITY)
(id:location-ilo.bvnode2-bvnode2--INFINITY)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Ordering Constraints:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> start dlm-clone then
start clvmd-clone (kind:Mandatory)
(id:order-dlm-clone-clvmd-clone-mandatory)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> start clvmd-clone then
start cluster-config-clone (kind:Mandatory)
(id:order-clvmd-clone-cluster-config-clone-mandatory)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> start
cluster-config-clone then start libvirtd-clone
(kind:Mandatory)
(id:order-cluster-config-clone-libvirtd-clone-mandatory)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> stop vm_smartbv2 then
stop libvirtd-clone (kind:Mandatory) (non-symmetrical)
(id:order-vm_smartbv2-libvirtd-clone-mandatory)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> stop vm_smartbv1 then
stop libvirtd-clone (kind:Mandatory) (non-symmetrical)
(id:order-vm_smartbv1-libvirtd-clone-mandatory)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> start libvirtd-clone
then start vm_smartbv2 (kind:Optional) (non-symmetrical)
(id:order-libvirtd-clone-vm_smartbv2-Optional)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> start libvirtd-clone
then start vm_smartbv1 (kind:Optional) (non-symmetrical)
(id:order-libvirtd-clone-vm_smartbv1-Optional)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Colocation Constraints:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> clvmd-clone with
dlm-clone (score:INFINITY)
(id:colocation-clvmd-clone-dlm-clone-INFINITY)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> cluster-config-clone
with clvmd-clone (score:INFINITY)
(id:colocation-cluster-config-clone-clvmd-clone-INFINITY)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> libvirtd-clone with
cluster-config-clone (score:INFINITY)
(id:colocation-libvirtd-clone-cluster-config-clone-INFINITY)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> vm_smartbv1 with
libvirtd-clone (score:INFINITY)
(id:colocation-vm_smartbv1-libvirtd-clone-INFINITY)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> vm_smartbv2 with
libvirtd-clone (score:INFINITY)
(id:colocation-vm_smartbv2-libvirtd-clone-INFINITY)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Resources Defaults:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">No defaults set<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Operations Defaults:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">No defaults set<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Cluster Properties:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">cluster-infrastructure:
corosync<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">cluster-name:
smartbvcluster<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">dc-version:
1.1.13-10.el7_2.4-44eb2dd<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">have-watchdog: false<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">last-lrm-refresh:
1476099872<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">maintenance-mode: false<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">no-quorum-policy: freeze<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">start-failure-is-fatal:
false<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"></span>stonith-enabled:
true<o:p></o:p></p>
<p class="MsoNormal"><o:p>==== <br>
</o:p></p>
<font face="Arial" color="Gray" size="1">
</font>
</div>
</div>
</body>
</html>