<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-html40/strict.dtd">
<html><head><meta name="qrichtext" content="1" /><style type="text/css">
p, li { white-space: pre-wrap; }
</style></head><body style=" font-family:'Noto Sans'; font-size:10pt; font-weight:400; font-style:normal;">
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Hi,</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">first question is why did you use tgt instead LIO? LIO is more common nowadays..<br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">And you need a colocations and ordering constraints.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Here is my config with lio-t (but I guess here is something wrong in the ressource-agents, but I will take a deeper look tomorrow):</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">pcs config</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Cluster Name: zfs-vmstorage</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Corosync Nodes:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> zfs-serv3 zfs-serv4</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Pacemaker Nodes:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> zfs-serv3 zfs-serv4</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Resources:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource: ha-ip (class=ocf provider=heartbeat type=IPaddr2)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Attributes: ip=192.168.2.10 cidr_netmask=24 nic=bond0</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Meta Attrs: target-role=Started </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Operations: start interval=0s timeout=20s (ha-ip-start-0s)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> stop interval=0s timeout=20s (ha-ip-stop-0s)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> monitor interval=10s timeout=20s (ha-ip-monitor-10s)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource: vm_storage (class=ocf provider=heartbeat type=ZFS)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Attributes: pool=vm_storage importargs="-d /dev/disk/by-vdev/"</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Meta Attrs: target-role=Started </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Operations: monitor interval=5s timeout=30s (vm_storage-monitor-5s)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> start interval=0s timeout=90 (vm_storage-start-0s)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> stop interval=0s timeout=90 (vm_storage-stop-0s)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource: iscsi-server (class=ocf provider=heartbeat type=iSCSITarget)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Attributes: implementation=lio-t iqn=iqn.2003-01.org.linux-iscsi.vm-storage.x8664:sn.1decabcxxxx. portals=192.168.2.10:3260 allowed_initiators="iqn.1998-01.com.vmware:brainslug9-7548xxxx iqn.1998-01.com.vmware:brainslug8-058xxxx iqn.1998-01.com.vmware:brainslug7-592bxxxx iqn.1998-01.com.vmware:brainslug10-5564cxxxx"</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource: iscsi-lun0 (class=ocf provider=heartbeat type=iSCSILogicalUnit)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Attributes: implementation=lio-t target_iqn=iqn.2003-01.org.linux-iscsi.vm-storage.x8664:sn.1decabcxxxx. lun=0 path=/dev/zvol/vm_storage/zfs-vol1</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource: iscsi-lun1 (class=ocf provider=heartbeat type=iSCSILogicalUnit)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Attributes: implementation=lio-t target_iqn=iqn.2003-01.org.linux-iscsi.vm-storage.x8664:sn.1decabcxxxx. lun=1 path=/dev/zvol/vm_storage/zfs-vol2</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource: iscsi-lun2 (class=ocf provider=heartbeat type=iSCSILogicalUnit)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Attributes: implementation=lio-t target_iqn=iqn.2003-01.org.linux-iscsi.vm-storage.x8664:sn.1decabcxxxx. lun=2 path=/dev/zvol/vm_storage/zfs-vol3</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Stonith Devices:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource: resIPMI-zfs4 (class=stonith type=external/ipmi)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Attributes: hostname=zfs-serv4 ipaddr=172.xx.xx.xx userid=USER passwd=SECRET interface=lan priv=OPERATOR pcmk_delay_max=20</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Operations: monitor interval=60s (resIPMI-zfs4-monitor-60s)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource: resIPMI-zfs3 (class=stonith type=external/ipmi)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Attributes: hostname=zfs-serv3 ipaddr=172.xx.xx.xx userid=user passwd=SECRET interface=lan priv=OPERATOR pcmk_delay_max=20</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Operations: monitor interval=60s (resIPMI-zfs3-monitor-60s)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Fencing Levels:</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Location Constraints:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource: resIPMI-zfs3</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Disabled on: zfs-serv3 (score:-INFINITY) (id:location-resIPMI-zfs3-zfs-serv3--INFINITY)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource: resIPMI-zfs4</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Disabled on: zfs-serv4 (score:-INFINITY) (id:location-resIPMI-zfs4-zfs-serv4--INFINITY)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Ordering Constraints:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource Sets:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> set ha-ip iscsi-lun0 iscsi-lun1 iscsi-lun2 iscsi-server vm_storage action=stop (id:pcs_rsc_order_set_ha-ip_iscsi-server_vm_storage-1) setoptions symmetrical=false (id:pcs_rsc_order_set_ha-ip_iscsi-server_vm_storage)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> set vm_storage iscsi-server iscsi-lun0 iscsi-lun1 iscsi-lun2 ha-ip action=start (id:pcs_rsc_order_set_iscsi-server_vm_storage_ha-ip-1) setoptions symmetrical=false (id:pcs_rsc_order_set_iscsi-server_vm_storage_ha-ip)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Colocation Constraints:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Resource Sets:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> set ha-ip vm_storage iscsi-server iscsi-lun0 iscsi-lun1 iscsi-lun2 (id:pcs_rsc_colocation_set_ha-ip_vm_storage_iscsi-server-1) setoptions score=INFINITY (id:pcs_rsc_colocation_set_ha-ip_vm_storage_iscsi-server)</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Ticket Constraints:</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Alerts:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> No alerts defined</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Resources Defaults:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> resource-stickiness: 100</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Operations Defaults:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> No defaults set</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Cluster Properties:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> cluster-infrastructure: corosync</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> cluster-name: zfs-vmstorage</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> dc-version: 1.1.16-94ff4df</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> have-watchdog: false</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> last-lrm-refresh: 1540199247</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> no-quorum-policy: stop</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> stonith-enabled: true</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Quorum:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> Options:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /><br /><br />On Wednesday, October 24, 2018 8:59:01 AM CEST LiFeng Zhang wrote:<br /></p>
<p style=" margin-top:12px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Hello Friends, </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">further thought about this situation: </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">I want to have a cluster service tgtd, on a "primary/second" DRBD, since the DRBD is only active on the primary node, so the tgtd won't success on the secondary node. </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">So should i, or how do config a primary/seconday tgtd service? <br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Or, which feature from pacemaker should i use, so the tgtd starts only on one node? </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">for any suggestions, thank you very much in advance </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Best Regards </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Lifeng<br /></p>
<p style=" margin-top:12px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Hello Dear Andrei Borzenkov, </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Thank you very much for your answer. I've check the logs all the time, but there are nothing helpful , just a bunch of heartbeat messages. </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Anyway, i've read the book "Packt - CentOS High Availability" published in 2015, and got some new ideas, and tried out, the situation is something new. </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<hr width="100%"/>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">pcs resource create p_iSCSITarget ocf:heartbeat:iSCSITarget implementation="tgt" iqn="iqn.2018-08.s-ka.local:disk" tid="1"<br />pcs resource create p_iSCSILogicalUnit ocf:heartbeat:iSCSILogicalUnit implementation="tgt" target_iqn="iqn.2018-08.s-ka.local:disk" lun="10" path="/dev/drbd/by-disk/vg0/ipstor0"<br />pcs resource group add p_iSCSI ClusterIP p_iSCSITarget p_iSCSILogicalUnit <br />pcs constraint colocation set ClusterIP p_iSCSITarget p_iSCSILogicalUnit<br /></p>
<hr width="100%"/>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">The difference from previous version is here: use iqn "iqn.2018-08.s-ka.local:disk" instead of "iqn.2018-08.s-ka.local:disk.1", which the last ".1" maybe means the "tid". </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">now i have new problem, because the resource and tgtd are startet, although i set "colocation constraint", the pacemaker always try to start tgtd on another node. </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">how to i solve this? thank you people in advance! </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">here the output from "pcs status": </p>
<hr width="100%"/>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">[root@drbd0 /]# pcs status<br />Cluster name: cluster1<br />Stack: corosync<br />Current DC: drbd0-ha.s-ka.local (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum<br />Last updated: Wed Oct 24 08:43:29 2018<br />Last change: Wed Oct 24 08:43:24 2018 by root via cibadmin on drbd0-ha.s-ka.local<br /><br />2 nodes configured<br />5 resources configured<br /><br />Online: [ drbd0-ha.s-ka.local drbd1-ha.s-ka.local ]<br /><br />Full list of resources:<br /><br /> Master/Slave Set: ipstor0Clone [ipstor0]<br /> Masters: [ drbd0-ha.s-ka.local ]<br /> Slaves: [ drbd1-ha.s-ka.local ]<br /> Resource Group: p_iSCSI<br /> ClusterIP (ocf::heartbeat:IPaddr2): Started drbd0-ha.s-ka.local<br /> p_iSCSITarget (ocf::heartbeat:iSCSITarget): Started drbd0-ha.s-ka.local<br /> p_iSCSILogicalUnit (ocf::heartbeat:iSCSILogicalUnit): Started drbd0-ha.s-ka.local<br /><br />Failed Actions:<br />* p_iSCSITarget_start_0 on drbd1-ha.s-ka.local 'unknown error' (1): call=32, status=complete, exitreason='',<br /> last-rc-change='Wed Oct 24 08:37:25 2018', queued=0ms, exec=23ms<br />* p_iSCSILogicalUnit_start_0 on drbd1-ha.s-ka.local 'unknown error' (1): call=38, status=complete, exitreason='',<br /> last-rc-change='Wed Oct 24 08:37:55 2018', queued=0ms, exec=28ms<br /><br /><br />Daemon Status:<br /> corosync: active/enabled<br /> pacemaker: active/enabled<br /> pcsd: active/enabled<br />[root@drbd0 /] </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">[root@drbd0 /]# pcs constraint show --full<br />Location Constraints:<br />Ordering Constraints:<br />Colocation Constraints:<br /> Resource Sets:<br /> set ClusterIP p_iSCSITarget p_iSCSILogicalUnit (id:pcs_rsc_set_ClusterIP_p_iSCSITarget_p_iSCSILogicalUnit) setoptions score=INFINITY (id:pcs_rsc_colocation_set_ClusterIP_p_iSCSITarget_p_iSCSILogicalUnit)<br />Ticket Constraints:<br />[root@drbd0 /]# <br /></p>
<hr width="100%"/>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Best Regards </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Lifeng<br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">在 2018/10/19 06:02, Andrei Borzenkov 写道:<br /></p>
<pre style=" margin-top:12px; margin-bottom:12px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">16.10.2018 15:29, LiFeng Zhang пишет:</span> </pre>
<pre style=" margin-top:12px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Hi, all dear friends,</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">i need your help to enable the hot switch of iSCSI under a</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Pacemaker/Corosync Cluster, which has a iSCSI Device based on a two node</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">DRBD Replication.</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">I've got the Pacemaker/Corosync cluster working, DRBD replication also</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">working, but it stuck at iSCSI, i can manually start a tgtd on one node,</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">so the VCSA can recognize the iSCSI Disk and create VMFS/StorageObject</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">on it, and then i can create a test VM on that VMFS.</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">But when i switch the Primary/Secondary of DRBD, although the test VM</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">still running, but the underlying Disk became read-only. As far as i</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">know, the tgtd should be handled by Pacemaker so it will automatically</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">start on the Primary DRBD Instance, but in my situation it's sadly NOT.</span></pre>
<pre style=" margin-top:0px; margin-bottom:12px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> </pre>
<pre style=" margin-top:12px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">pacemaker only handles resources that were started by pacemaker.</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">According to your output below, in all cases resource was stopped from</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">pacemaker point of view and all pacemaker attempts to start resource</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">failed. You should troubleshoot why they failed. This requires knowledge</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">of specific resource agent, sadly I am not familiar with iSCSI target.</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">pacemaker logs may include more information from resource agent than</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">just "unknown reason".</span></pre>
<pre style=" margin-top:0px; margin-bottom:12px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> </pre>
<pre style=" margin-top:12px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">I've tried all kinds of resources/manuals/documents, but they all mixed</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">with extra information, other system, other software version.</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">And one of my BEST reference (the closest configuration to mein) is this</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">url: </span><a href="https://nnc3.com/mags/LJ_1994-2014/LJ/217/11275.html"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">https://nnc3.com/mags/LJ_1994-2014/LJ/217/11275.html</span></a></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack'; text-decoration: underline; color:#2980b9;"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">The difference betwee me and this article, i think is i don't have LVM</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Volume but only raw iSCSI Disk, and i have to translate CRM commands</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">into PCS commands</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">But after i "copied" the configuration from this article, my cluster can</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">not start anymore, i've tried remove the LVM resource (which caused a</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">"device not found" error), but the resource group still can't start and</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">without any explicit "reason" from Pacemaker.</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">*1*. The whole configuration is under a two node ESXi 6.5 Cluster, which</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">has a VCSA one one ESXi host installed.</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">I have a simple diagram in attachment, which may state the deployment</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">better.</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">2. start point:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">The involved hosts are all with mapped through local dns, which also</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">includes the floating vip, the local domain is s-ka.local:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">firwall: fw01.s-ka.local. IN A 192.168.95.249</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">vcsa: vc01.s-ka.local. IN A 192.168.95.30</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">esxi: esx01.s-ka.local. IN A 192.168.95.5</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">esxi: esx02.s-ka.local. IN A 192.168.95.7</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">drbd: drbd0.s-ka.local. IN A 192.168.95.45</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">drbd: drbd1.s-ka.local. IN A 192.168.95.47</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">vip: ipstor0.s-ka.local. IN A 192.168.95.48</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">heartbeat: drbd0-ha.s-ka.local. IN A 192.168.96.45</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">heartbeat: drbd1-ha.s-ka.local. IN A 192.168.96.47</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">The both drbd server are CentOS 7.5, the installed packages are here:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 ~]# cat /etc/centos-release</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">CentOS Linux release 7.5.1804 (Core)</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 ~]# uname -a</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Linux drbd0.s-ka.local 3.10.0-862.9.1.el7.x86_64 #1 SMP Mon Jul 16</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">16:29:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# yum list installed|grep pacemaker</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">pacemaker.x86_64 1.1.18-11.el7_5.3 @updates</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">pacemaker-cli.x86_64 1.1.18-11.el7_5.3 @updates</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">pacemaker-cluster-libs.x86_64 1.1.18-11.el7_5.3 @updates</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">pacemaker-libs.x86_64 1.1.18-11.el7_5.3 @updates</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# yum list installed|grep coro</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">corosync.x86_64 2.4.3-2.el7_5.1 @updates</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">corosynclib.x86_64 2.4.3-2.el7_5.1 @updates</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# yum list installed|grep drbd</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">drbd90-utils.x86_64 9.3.1-1.el7.elrepo @elrepo</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">kmod-drbd90.x86_64 9.0.14-1.el7_5.elrepo @elrepo</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# yum list installed|grep -i scsi</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">lsscsi.x86_64 0.27-6.el7 @anaconda</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">scsi-target-utils.x86_64 1.0.55-4.el7 @epel</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3. configurations</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.1 ok first the drbd configuration</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# cat /etc/drbd.conf</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"># You can find an example in /usr/share/doc/drbd.../drbd.conf.example</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">include "drbd.d/global_common.conf";</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">include "drbd.d/*.res";</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# cat /etc/drbd.d/r0.res</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">resource iscsivg01 {</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> protocol C;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> device /dev/drbd0;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> disk /dev/vg0/ipstor0;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> flexible-meta-disk internal;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> on drbd0.s-ka.local {</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> #volume 0 {</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> #device /dev/drbd0;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> #disk /dev/vg0/ipstor0;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> #flexible-meta-disk internal;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> #}</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> address 192.168.96.45:7788;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> }</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> on drbd1.s-ka.local {</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> #volume 0 {</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> #device /dev/drbd0;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> #disk /dev/vg0/ipstor0;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> #flexible-meta-disk internal;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> #}</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> address 192.168.96.47:7788;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> }</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">}</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.2 then the drbd device</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# lsblk</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">sda 8:0 0 25G 0 disk</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">├─sda1 8:1 0 1G 0 part /boot</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">└─sda2 8:2 0 24G 0 part</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> ├─centos-root 253:0 0 22G 0 lvm /</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> └─centos-swap 253:1 0 2G 0 lvm [SWAP]</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">sdb 8:16 0 500G 0 disk</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">└─sdb1 8:17 0 500G 0 part</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> └─vg0-ipstor0 253:2 0 500G 0 lvm</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> └─drbd0 147:0 0 500G 1 disk</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">sr0 11:0 1 1024M 0 rom</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# tree /dev/drbd</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">drbd/ drbd0</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# tree /dev/drbd</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">/dev/drbd</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">├── by-disk</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">│ └── vg0</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">│ └── ipstor0 -> ../../../drbd0</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">└── by-res</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> └── iscsivg01</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> └── 0 -> ../../../drbd0</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">4 directories, 2 files</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.3drbd status</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# drbdadm status</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">iscsivg01 role:Secondary</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> disk:UpToDate</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> drbd0.s-ka.local role:Primary</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> peer-disk:UpToDate</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 ~]# drbdadm status</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">iscsivg01 role:Primary</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> disk:UpToDate</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> drbd1.s-ka.local role:Secondary</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> peer-disk:UpToDate</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 ~]# cat /proc/drbd</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">version: 9.0.14-1 (api:2/proto:86-113)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">GIT-hash: 62f906cf44ef02a30ce0c148fec223b40c51c533 build by mockbuild@,</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">2018-05-04 03:32:42</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Transports (api:16): tcp (9.0.14-1)</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.4 Corosync configuration</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# cat /etc/corosync/corosync.conf</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">totem {</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> version: 2</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> cluster_name: cluster1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> secauth: off</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> transport: udpu</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">}</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">nodelist {</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> node {</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> ring0_addr: drbd0-ha.s-ka.local</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> nodeid: 1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> }</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> node {</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> ring0_addr: drbd1-ha.s-ka.local</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> nodeid: 2</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> }</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">}</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">quorum {</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> provider: corosync_votequorum</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> two_node: 1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">}</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">logging {</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> to_logfile: yes</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> logfile: /var/log/cluster/corosync.log</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> to_syslog: yes</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">}</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.5 Corosync status:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# systemctl status corosync</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">● corosync.service - Corosync Cluster Engine</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Loaded: loaded (/usr/lib/systemd/system/corosync.service; enabled;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">vendor preset: disabled)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Active: active (running) since Sun 2018-10-14 02:58:01 CEST; 2 days ago</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Docs: man:corosync</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> man:corosync.conf</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> man:corosync_overview</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Process: 1095 ExecStart=/usr/share/corosync/corosync start</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">(code=exited, status=0/SUCCESS)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Main PID: 1167 (corosync)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> CGroup: /system.slice/corosync.service</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> └─1167 corosync</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 14 02:58:00 drbd0.s-ka.local corosync[1167]: [MAIN ] Completed</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">service synchronization, ready to provide service.</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 14 02:58:01 drbd0.s-ka.local corosync[1095]: Starting Corosync</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Cluster Engine (corosync): [ OK ]</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 14 02:58:01 drbd0.s-ka.local systemd[1]: Started Corosync Cluster</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Engine.</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 14 10:46:03 drbd0.s-ka.local corosync[1167]: [TOTEM ] A new</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">membership (192.168.96.45:384) was formed. Members left: 2</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 14 10:46:03 drbd0.s-ka.local corosync[1167]: [QUORUM] Members[1]: 1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 14 10:46:03 drbd0.s-ka.local corosync[1167]: [MAIN ] Completed</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">service synchronization, ready to provide service.</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 14 10:46:22 drbd0.s-ka.local corosync[1167]: [TOTEM ] A new</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">membership (192.168.96.45:388) was formed. Members joined: 2</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 14 10:46:22 drbd0.s-ka.local corosync[1167]: [CPG ] downlist</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">left_list: 0 received in state 0</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 14 10:46:22 drbd0.s-ka.local corosync[1167]: [QUORUM] Members[2]: 1 2</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 14 10:46:22 drbd0.s-ka.local corosync[1167]: [MAIN ] Completed</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">service synchronization, ready to provide service.</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.6 tgtd configuration:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# cat /etc/tgt/targets.conf</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"># This is a sample config file for tgt-admin.</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">#</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"># The "#" symbol disables the processing of a line.</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"># Set the driver. If not specified, defaults to "iscsi".</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">default-driver iscsi</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"># Set iSNS parameters, if needed</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">#iSNSServerIP 192.168.111.222</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">#iSNSServerPort 3205</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">#iSNSAccessControl On</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">#iSNS On</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"># Continue if tgtadm exits with non-zero code (equivalent of</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"># --ignore-errors command line option)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">#ignore-errors yes</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"><target iqn.2018-08.s-ka.local:disk.1></span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> lun 10</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> backing-store /dev/drbd0</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> initiator-address 192.168.96.0/24</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> initiator-address 192.168.95.0/24</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> target-address 192.168.95.48</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"></target></span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.7 tgtd has been on both server disabled, only startable from current</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Primary DRBD Node.</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Secondary Node:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# systemctl status tgtd</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">● tgtd.service - tgtd iSCSI target daemon</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Loaded: loaded (/usr/lib/systemd/system/tgtd.service; disabled;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">vendor preset: disabled)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Active: inactive (dead)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd1 ~]# systemctl restart tgtd</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Job for tgtd.service failed because the control process exited with</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">error code. See "systemctl status tgtd.service" and "journalctl -xe" for</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">details.</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Primary Node:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# systemctl status tgtd</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">● tgtd.service - tgtd iSCSI target daemon</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Loaded: loaded (/usr/lib/systemd/system/tgtd.service; disabled;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">vendor preset: disabled)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Active: inactive (dead)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# systemctl restart tgtd</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# systemctl status tgtd</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">● tgtd.service - tgtd iSCSI target daemon</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Loaded: loaded (/usr/lib/systemd/system/tgtd.service; disabled;</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">vendor preset: disabled)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Active: active (running) since Tue 2018-10-16 14:09:47 CEST; 2min 29s</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">ago</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Process: 22300 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">--name State -v ready (code=exited, status=0/SUCCESS)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Process: 22272 ExecStartPost=/usr/sbin/tgt-admin -e -c $TGTD_CONFIG</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">(code=exited, status=0/SUCCESS)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Process: 22271 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">--name State -v offline (code=exited, status=0/SUCCESS)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Process: 22270 ExecStartPost=/bin/sleep 5 (code=exited, status=0/SUCCESS)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Main PID: 22269 (tgtd)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> CGroup: /system.slice/tgtd.service</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> └─22269 /usr/sbin/tgtd -f</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 16 14:09:42 drbd0.s-ka.local systemd[1]: Starting tgtd iSCSI target</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">daemon...</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 16 14:09:42 drbd0.s-ka.local tgtd[22269]: tgtd: iser_ib_init(3436)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Failed to initialize RDMA; load kernel modules?</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 16 14:09:42 drbd0.s-ka.local tgtd[22269]: tgtd:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">work_timer_start(146) use timer_fd based scheduler</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 16 14:09:42 drbd0.s-ka.local tgtd[22269]: tgtd:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">bs_init_signalfd(267) could not open backing-store module directory</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">/usr/lib64/tgt/backing-store</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 16 14:09:42 drbd0.s-ka.local tgtd[22269]: tgtd: bs_init(386) use</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">signalfd notification</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 16 14:09:47 drbd0.s-ka.local tgtd[22269]: tgtd: device_mgmt(246)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">sz:16 params:path=/dev/drbd0</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 16 14:09:47 drbd0.s-ka.local tgtd[22269]: tgtd: bs_thread_open(408) 16</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Oct 16 14:09:47 drbd0.s-ka.local systemd[1]: Started tgtd iSCSI target</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">daemon.</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.8 it was until this point all working, but if i switched the DRBD</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Primary Node, it won't work anymore (FileSystem of test Node became</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">read-only)</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">so i changed the pcs configuration according to the previously mentioned</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">article:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style=" margin-top:0px; margin-bottom:12px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> </pre>
<pre style=" margin-top:12px; margin-bottom:12px; margin-left:200px; margin-right:200px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">pcs resource create p_iscsivg01 ocf:heartbeat:LVM volgrpname="vg0" op</span> </pre>
<pre style=" margin-top:12px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">monitor interval="30"</span></pre>
<pre style=" margin-top:0px; margin-bottom:12px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> </pre>
<pre style=" margin-top:12px; margin-bottom:12px; margin-left:200px; margin-right:200px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">pcs resource group add p_iSCSI p_iscsivg01 p_iSCSITarget</span> </pre>
<pre style=" margin-top:12px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">p_iSCSILogicalUnit ClusterIP</span></pre>
<pre style=" margin-top:0px; margin-bottom:12px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> </pre>
<pre style=" margin-top:12px; margin-bottom:12px; margin-left:200px; margin-right:200px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">pcs constraint order start ipstor0Clone then start p_iSCSI then start</span> </pre>
<pre style=" margin-top:12px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">ipstor0Clone:Master</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 ~]# pcs status</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Cluster name: cluster1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Stack: corosync</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Current DC: drbd0-ha.s-ka.local (version</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Last updated: Sun Oct 14 01:38:18 2018</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Last change: Sun Oct 14 01:37:58 2018 by root via cibadmin on</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">drbd0-ha.s-ka.local</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> 2 nodes configured</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> 6 resources configured</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Online: [ drbd0-ha.s-ka.local drbd1-ha.s-ka.local ]</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Full list of resources:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Master/Slave Set: ipstor0Clone [ipstor0]</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Masters: [ drbd0-ha.s-ka.local ]</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Slaves: [ drbd1-ha.s-ka.local ]</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Resource Group: p_iSCSI</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> p_iscsivg01 (ocf::heartbeat:LVM): Stopped</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> p_iSCSITarget (ocf::heartbeat:iSCSITarget): Stopped</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> p_iSCSILogicalUnit (ocf::heartbeat:iSCSILogicalUnit): Stopped</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> ClusterIP (ocf::heartbeat:IPaddr2): Stopped</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Failed Actions:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> * p_iSCSILogicalUnit_start_0 on drbd0-ha.s-ka.local 'unknown error'</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">(1): call=42, status=complete, exitreason='',</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> last-rc-change='Sun Oct 14 01:20:38 2018', queued=0ms, exec=28ms</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> * p_iSCSITarget_start_0 on drbd0-ha.s-ka.local 'unknown error' (1):</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">call=40, status=complete, exitreason='',</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> last-rc-change='Sun Oct 14 00:54:36 2018', queued=0ms, exec=23ms</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> * p_iscsivg01_start_0 on drbd0-ha.s-ka.local 'unknown error' (1):</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">call=48, status=complete, exitreason='Volume group [iscsivg01] does not</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">exist or contains error! Volume group "iscsivg01" not found',</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> last-rc-change='Sun Oct 14 01:32:49 2018', queued=0ms, exec=47ms</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> * p_iSCSILogicalUnit_start_0 on drbd1-ha.s-ka.local 'unknown error'</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">(1): call=41, status=complete, exitreason='',</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> last-rc-change='Sun Oct 14 01:20:38 2018', queued=0ms, exec=31ms</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> * p_iSCSITarget_start_0 on drbd1-ha.s-ka.local 'unknown error' (1):</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">call=39, status=complete, exitreason='',</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> last-rc-change='Sun Oct 14 00:54:36 2018', queued=0ms, exec=24ms</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> * p_iscsivg01_start_0 on drbd1-ha.s-ka.local 'unknown error' (1):</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">call=47, status=complete, exitreason='Volume group [iscsivg01] does not</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">exist or contains error! Volume group "iscsivg01" not found',</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> last-rc-change='Sun Oct 14 01:32:49 2018', queued=0ms, exec=50ms</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Daemon Status:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> corosync: active/enabled</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> pacemaker: active/enabled</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> pcsd: active/enabled</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> [root@drbd0 ~]#</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.9 since the "device not found" error, so i remove the LVM, it looks</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">like this now:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">actually it was changed between /dev/drbd/by-disk and /dev/drbd/by-res,</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">but no effects</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# pcs status</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Cluster name: cluster1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Stack: corosync</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Current DC: drbd0-ha.s-ka.local (version 1.1.18-11.el7_5.3-2b07d5c5a9) -</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">partition with quorum</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Last updated: Tue Oct 16 14:18:09 2018</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Last change: Sun Oct 14 02:06:36 2018 by root via cibadmin on</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">drbd0-ha.s-ka.local</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">2 nodes configured</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">5 resources configured</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Online: [ drbd0-ha.s-ka.local drbd1-ha.s-ka.local ]</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Full list of resources:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Master/Slave Set: ipstor0Clone [ipstor0]</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Masters: [ drbd0-ha.s-ka.local ]</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Slaves: [ drbd1-ha.s-ka.local ]</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Resource Group: p_iSCSI</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> p_iSCSITarget (ocf::heartbeat:iSCSITarget): Stopped</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> p_iSCSILogicalUnit (ocf::heartbeat:iSCSILogicalUnit): Stopped</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> ClusterIP (ocf::heartbeat:IPaddr2): Stopped</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Failed Actions:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">* p_iSCSITarget_start_0 on drbd0-ha.s-ka.local 'unknown error' (1):</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">call=12, status=complete, exitreason='',</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> last-rc-change='Sun Oct 14 02:58:04 2018', queued=1ms, exec=58ms</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">* p_iSCSITarget_start_0 on drbd1-ha.s-ka.local 'unknown error' (1):</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">call=12, status=complete, exitreason='',</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> last-rc-change='Sun Oct 14 10:47:06 2018', queued=0ms, exec=22ms</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Daemon Status:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> corosync: active/enabled</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> pacemaker: active/enabled</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> pcsd: active/enabled</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]#</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.10 i've tried with "pcs resouce debug-start xxx --full" on the DRBD</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Primary Node,</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# pcs resource debug-start p_iSCSI --full</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Error: unable to debug-start a group, try one of the group's resource(s)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">(p_iSCSITarget,p_iSCSILogicalUnit,ClusterIP)</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# pcs resource debug-start p_iSCSITarget --full</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Operation start for p_iSCSITarget (ocf:heartbeat:iSCSITarget) returned:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">'ok' (0)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> > stderr: DEBUG: p_iSCSITarget start : 0</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# pcs resource debug-start p_iSCSILogicalUnit --full</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Operation start for p_iSCSILogicalUnit (ocf:heartbeat:iSCSILogicalUnit)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">returned: 'unknown error' (1)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> > stderr: ERROR: tgtadm: this logical unit number already exists</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# pcs resource debug-start ClusterIP --full</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Operation start for ClusterIP (ocf:heartbeat:IPaddr2) returned: 'ok' (0)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> > stderr: INFO: Adding inet address 192.168.95.48/32 with broadcast</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">address 192.168.95.255 to device ens192</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> > stderr: INFO: Bringing device ens192 up</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> > stderr: INFO: /usr/libexec/heartbeat/send_arp -i 200 -c 5 -p</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">/var/run/resource-agents/send_arp-192.168.95.48 -I ens192 -m auto</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">192.168.95.48</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]#</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.11 as you may seen, there are errors, but "p_iSCSITarget" was</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">successfully startet. but "pcs status" show still "stopped"</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# pcs status</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Cluster name: cluster1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Stack: corosync</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Current DC: drbd0-ha.s-ka.local (version 1.1.18-11.el7_5.3-2b07d5c5a9) -</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">partition with quorum</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Last updated: Tue Oct 16 14:22:38 2018</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Last change: Sun Oct 14 02:06:36 2018 by root via cibadmin on</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">drbd0-ha.s-ka.local</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">2 nodes configured</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">5 resources configured</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Online: [ drbd0-ha.s-ka.local drbd1-ha.s-ka.local ]</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Full list of resources:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Master/Slave Set: ipstor0Clone [ipstor0]</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Masters: [ drbd0-ha.s-ka.local ]</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Slaves: [ drbd1-ha.s-ka.local ]</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Resource Group: p_iSCSI</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> p_iSCSITarget (ocf::heartbeat:iSCSITarget): Stopped</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> p_iSCSILogicalUnit (ocf::heartbeat:iSCSILogicalUnit): Stopped</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> ClusterIP (ocf::heartbeat:IPaddr2): Stopped</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Failed Actions:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">* p_iSCSITarget_start_0 on drbd0-ha.s-ka.local 'unknown error' (1):</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">call=12, status=complete, exitreason='',</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> last-rc-change='Sun Oct 14 02:58:04 2018', queued=1ms, exec=58ms</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">* p_iSCSITarget_start_0 on drbd1-ha.s-ka.local 'unknown error' (1):</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">call=12, status=complete, exitreason='',</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> last-rc-change='Sun Oct 14 10:47:06 2018', queued=0ms, exec=22ms</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Daemon Status:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> corosync: active/enabled</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> pacemaker: active/enabled</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> pcsd: active/enabled</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]#</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">3.12 the pcs config is:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]# pcs config</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Cluster Name: cluster1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Corosync Nodes:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> drbd0-ha.s-ka.local drbd1-ha.s-ka.local</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Pacemaker Nodes:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> drbd0-ha.s-ka.local drbd1-ha.s-ka.local</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Resources:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Master: ipstor0Clone</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Meta Attrs: master-node-max=1 clone-max=2 notify=true master-max=1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">clone-node-max=1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Resource: ipstor0 (class=ocf provider=linbit type=drbd)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Attributes: drbd_resource=iscsivg01</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Operations: demote interval=0s timeout=90 (ipstor0-demote-interval-0s)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> monitor interval=60s (ipstor0-monitor-interval-60s)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> notify interval=0s timeout=90 (ipstor0-notify-interval-0s)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> promote interval=0s timeout=90 (ipstor0-promote-interval-0s)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> reload interval=0s timeout=30 (ipstor0-reload-interval-0s)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> start interval=0s timeout=240 (ipstor0-start-interval-0s)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> stop interval=0s timeout=100 (ipstor0-stop-interval-0s)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Group: p_iSCSI</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Resource: p_iSCSITarget (class=ocf provider=heartbeat type=iSCSITarget)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Attributes: implementation=tgt iqn=iqn.2018-08.s-ka.local:disk.1 tid=1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Operations: monitor interval=30 timeout=60</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">(p_iSCSITarget-monitor-interval-30)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> start interval=0 timeout=60 (p_iSCSITarget-start-interval-0)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> stop interval=0 timeout=60 (p_iSCSITarget-stop-interval-0)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Resource: p_iSCSILogicalUnit (class=ocf provider=heartbeat</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">type=iSCSILogicalUnit)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Attributes: implementation=tgt lun=10</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">path=/dev/drbd/by-disk/vg0/ipstor0 target_iqn=iqn.2018-08.s-ka.local:disk.1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Operations: monitor interval=30 timeout=60</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">(p_iSCSILogicalUnit-monitor-interval-30)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> start interval=0 timeout=60</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">(p_iSCSILogicalUnit-start-interval-0)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> stop interval=0 timeout=60</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">(p_iSCSILogicalUnit-stop-interval-0)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Resource: ClusterIP (class=ocf provider=heartbeat type=IPaddr2)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Attributes: cidr_netmask=32 ip=192.168.95.48</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Operations: monitor interval=30s (ClusterIP-monitor-interval-30s)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> start interval=0s timeout=20s (ClusterIP-start-interval-0s)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> stop interval=0s timeout=20s (ClusterIP-stop-interval-0s)</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Stonith Devices:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Fencing Levels:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Location Constraints:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Ordering Constraints:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> start ipstor0Clone then start p_iSCSI (kind:Mandatory)</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Colocation Constraints:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Ticket Constraints:</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Alerts:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> No alerts defined</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Resources Defaults:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> migration-threshold: 1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Operations Defaults:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> No defaults set</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Cluster Properties:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> cluster-infrastructure: corosync</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> cluster-name: cluster1</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> dc-version: 1.1.18-11.el7_5.3-2b07d5c5a9</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> have-watchdog: false</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> last-lrm-refresh: 1539474248</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> no-quorum-policy: ignore</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> stonith-enabled: false</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Quorum:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';"> Options:</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">[root@drbd0 corosync]#</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">------------------------------------------------------------------------</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">4. so i am out of hands. don't what to do, may just dive into</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">pacemaker's source code??</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Hope to get any feedback or tips from you, thank you very much in</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">advance :)</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Best Regards</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Zhang</span></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack';"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">_______________________________________________</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Users mailing list: </span><a href="mailto:Users@clusterlabs.org"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">Users@clusterlabs.org</span></a></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><a href="https://lists.clusterlabs.org/mailman/listinfo/users"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">https://lists.clusterlabs.org/mailman/listinfo/users</span></a></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; font-family:'Hack'; text-decoration: underline; color:#2980b9;"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Project Home: </span><a href="http://www.clusterlabs.org"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">http://www.clusterlabs.org</span></a></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Getting started: </span><a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</span></a></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Bugs: </span><a href="http://bugs.clusterlabs.org"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">http://bugs.clusterlabs.org</span></a></pre>
<pre style=" margin-top:0px; margin-bottom:12px; margin-left:160px; margin-right:160px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> </pre>
<pre style=" margin-top:12px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">_______________________________________________</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Users mailing list: </span><a href="mailto:Users@clusterlabs.org"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">Users@clusterlabs.org</span></a></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><a href="https://lists.clusterlabs.org/mailman/listinfo/users"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">https://lists.clusterlabs.org/mailman/listinfo/users</span></a></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; font-family:'Hack'; text-decoration: underline; color:#2980b9;"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Project Home: </span><a href="http://www.clusterlabs.org"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">http://www.clusterlabs.org</span></a></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Getting started: </span><a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</span></a></pre>
<pre style=" margin-top:0px; margin-bottom:12px; margin-left:120px; margin-right:120px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Bugs: </span><a href="http://bugs.clusterlabs.org"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">http://bugs.clusterlabs.org</span></a> </pre>
<p style=" margin-top:12px; margin-bottom:12px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /><br /></p>
<pre style=" margin-top:12px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">_______________________________________________</span></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Users mailing list: </span><a href="mailto:Users@clusterlabs.org"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">Users@clusterlabs.org</span></a></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><a href="https://lists.clusterlabs.org/mailman/listinfo/users"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">https://lists.clusterlabs.org/mailman/listinfo/users</span></a></pre>
<pre style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; font-family:'Hack'; text-decoration: underline; color:#2980b9;"><br /></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Project Home: </span><a href="http://www.clusterlabs.org"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">http://www.clusterlabs.org</span></a></pre>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Getting started: </span><a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</span></a></pre>
<pre style=" margin-top:0px; margin-bottom:12px; margin-left:80px; margin-right:80px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Hack';">Bugs: </span><a href="http://bugs.clusterlabs.org"><span style=" font-family:'Hack'; text-decoration: underline; color:#2980b9;">http://bugs.clusterlabs.org</span></a> </pre>
<p style=" margin-top:12px; margin-bottom:12px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /><br /></p></body></html>