<div dir="ltr">Hi,<div class="gmail_extra"><br><div class="gmail_quote">On Tue, Nov 8, 2016 at 7:54 PM, Ryan Anstey <span dir="ltr"><<a href="mailto:ryan@treasuremart.net" target="_blank">ryan@treasuremart.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I've been running a ceph cluster with pacemaker for a few months now. Everything has been working normally, but when I added a fourth node it won't work like the others, even though their OS is the same and the configs are all synced via salt. I also don't understand pacemaker that well since I followed a guide for it. If anyone could steer me in the right direction I would greatly appreciate it. Thank you!<div><div><br></div><div>- My resources only start if the new node is the only active node.</div><div>- Once started on new node, if they are moved back to one of the original nodes, it won't go back to the new one.</div><div>- My resources work 100% if I start them manually (without pacemaker).</div><div>- (In the logs/configs below, my resources are named "unifi", "rbd_unifi" being the main one that's not working.)<br></div></div></div></blockquote><div><br></div><div>looking at your configuration below I would start with setting these resources to be "managed":</div><div><br></div><div>fs_unifi</div><div>lxc_unifi</div><div><br></div><div>and to also start that resources.</div><div><br></div><div>crm resource manage ...</div><div>crm resource start ...</div><div><br></div><div>Regards,</div><div>Andreas</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div></div><div><br></div>Log when running cleaning up the resource on the NEW node:<br><br><div>Nov 08 09:25:20 h4 Filesystem(fs_unifi)[18044]: WARNING: Couldn't find device [/dev/rbd/rbd/unifi]. Expected /dev/??? to exist<br> Nov 08 09:25:20 h4 lrmd[3564]: notice: lxc_unifi_monitor_0:18018:<wbr>stderr [ unifi doesn't exist ]<br> Nov 08 09:25:20 h4 crmd[3567]: notice: Operation lxc_unifi_monitor_0: not running (node=h4, call=484, rc=7, cib-update=390, confirmed=true)<br> Nov 08 09:25:20 h4 crmd[3567]: notice: h4-lxc_unifi_monitor_0:484 [ unifi doesn't exist\n ]<br> Nov 08 09:25:20 h4 crmd[3567]: notice: Operation fs_unifi_monitor_0: not running (node=h4, call=480, rc=7, cib-update=391, confirmed=true)<br> Nov 08 09:25:20 h4 crmd[3567]: notice: Operation rbd_unifi_monitor_0: not running (node=h4, call=476, rc=7, cib-update=392, confirmed=true)<br><br>Log when running cleaning up the resource on the OLD node:  <br><br><div>Nov 08 09:21:18 h3 crmd[11394]: warning: No match for shutdown action on 167838209</div><div>Nov 08 09:21:18 h3 crmd[11394]: warning: No match for shutdown action on 167838210</div><div>Nov 08 09:21:18 h3 crmd[11394]: warning: No match for shutdown action on 167838212</div><div>Nov 08 09:21:18 h3 crmd[11394]: warning: No match for shutdown action on 167838209</div><div>Nov 08 09:21:18 h3 crmd[11394]: warning: No match for shutdown action on 167838209</div><div>Nov 08 09:21:18 h3 crmd[11394]: warning: No match for shutdown action on 167838210</div><div>Nov 08 09:21:18 h3 crmd[11394]: warning: No match for shutdown action on 167838212</div><div>Nov 08 09:21:18 h3 crmd[11394]: notice: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]</div><div>Nov 08 09:21:18 h3 crmd[11394]: warning: No match for shutdown action on 167838210</div><div>Nov 08 09:21:18 h3 crmd[11394]: warning: No match for shutdown action on 167838212</div><div>Nov 08 09:21:18 h3 cib[11389]: warning: A-Sync reply to crmd failed: No message of desired type</div><div>Nov 08 09:21:18 h3 crmd[11394]: warning: No match for shutdown action on 167838211</div><div>Nov 08 09:21:22 h3 crmd[11394]: notice: Notifications disabled</div><div>Nov 08 09:21:24 h3 crmd[11394]: notice: Notifications disabled</div><div>Nov 08 09:21:24 h3 crmd[11394]: warning: No match for shutdown action on 167838211</div><div>Nov 08 09:21:25 h3 crmd[11394]: notice: Notifications disabled</div><div>Nov 08 09:21:25 h3 crmd[11394]: warning: No match for shutdown action on 167838211</div><div>Nov 08 09:21:26 h3 pengine[11393]: notice: Start   rbd_unifi        (h3)</div><div>Nov 08 09:21:26 h3 pengine[11393]: notice: Start   fs_unifi        (h3)</div><div>Nov 08 09:21:26 h3 pengine[11393]: notice: Start   lxc_unifi        (h3)</div><div>Nov 08 09:21:26 h3 pengine[11393]: notice: Calculated Transition 119: /var/lib/pacemaker/pengine/pe-<wbr>input-739.bz2</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Processing graph 119 (ref=pe_calc-dc-1478625686-<wbr>648) derived from /var/lib/pacemaker/pengine/pe-<wbr>input-739.bz2</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 12: monitor rbd_unifi_monitor_0 on h4</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 9: monitor rbd_unifi_monitor_0 on h3 (local)</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 6: monitor rbd_unifi_monitor_0 on h2</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 3: monitor rbd_unifi_monitor_0 on h1</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 13: monitor fs_unifi_monitor_0 on h4</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 10: monitor fs_unifi_monitor_0 on h3 (local)</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 7: monitor fs_unifi_monitor_0 on h2</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 4: monitor fs_unifi_monitor_0 on h1</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 14: monitor lxc_unifi_monitor_0 on h4</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 11: monitor lxc_unifi_monitor_0 on h3 (local)</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 8: monitor lxc_unifi_monitor_0 on h2</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Initiating action 5: monitor lxc_unifi_monitor_0 on h1</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Operation fs_unifi_monitor_0: ok (node=h3, call=446, rc=0, cib-update=767, confirmed=true)</div><div>Nov 08 09:21:26 h3 crmd[11394]: warning: Action 10 (fs_unifi_monitor_0) on h3 failed (target: 7 vs. rc: 0): Error</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Transition aborted by fs_unifi_monitor_0 'create' on h3: Event failed (magic=0:0;10:119:7:2bc5b523-<wbr>b947-4035-b296-4c099646298b, cib=0.1445.8, source=match_graph_event:381, 0)</div><div>Nov 08 09:21:26 h3 crmd[11394]: warning: Action 10 (fs_unifi_monitor_0) on h3 failed (target: 7 vs. rc: 0): Error</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Operation lxc_unifi_monitor_0: ok (node=h3, call=450, rc=0, cib-update=768, confirmed=true)</div><div>Nov 08 09:21:26 h3 crmd[11394]: warning: Action 11 (lxc_unifi_monitor_0) on h3 failed (target: 7 vs. rc: 0): Error</div><div>Nov 08 09:21:26 h3 crmd[11394]: warning: Action 11 (lxc_unifi_monitor_0) on h3 failed (target: 7 vs. rc: 0): Error</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Operation rbd_unifi_monitor_0: ok (node=h3, call=442, rc=0, cib-update=769, confirmed=true)</div><div>Nov 08 09:21:26 h3 crmd[11394]: warning: Action 9 (rbd_unifi_monitor_0) on h3 failed (target: 7 vs. rc: 0): Error</div><div>Nov 08 09:21:26 h3 crmd[11394]: warning: Action 9 (rbd_unifi_monitor_0) on h3 failed (target: 7 vs. rc: 0): Error</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Transition 119 (Complete=13, Pending=0, Fired=0, Skipped=0, Incomplete=4, Source=/var/lib/pacemaker/<wbr>pengine/pe-input-739.bz2): Complete</div><div>Nov 08 09:21:26 h3 pengine[11393]: notice: Calculated Transition 120: /var/lib/pacemaker/pengine/pe-<wbr>input-740.bz2</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Processing graph 120 (ref=pe_calc-dc-1478625686-<wbr>661) derived from /var/lib/pacemaker/pengine/pe-<wbr>input-740.bz2</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: Transition 120 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/<wbr>pengine/pe-input-740.bz2): Complete</div><div>Nov 08 09:21:26 h3 crmd[11394]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]</div><div><br></div><div>Output of crm configure show:</div><div><br></div><div><div>node 167838209: h1 \</div><div>        attributes standby=off</div><div>node 167838210: h2 \</div><div>        attributes standby=off</div><div>node 167838211: h3 \</div><div>        attributes standby=off</div><div>node 167838212: h4 \</div><div>        attributes standby=off</div><div>rsc_template fs Filesystem \</div><div>        op start timeout=60s interval=0 \</div><div>        op stop timeout=60s interval=0</div><div>rsc_template kvm VirtualDomain \</div><div>        params hypervisor="qemu:///system" \</div><div>        meta allow-migrate=true \</div><div>        op start timeout=120s interval=0 \</div><div>        op stop timeout=120s interval=0 \</div><div>        op monitor timeout=30 interval=10 depth=0</div><div>rsc_template lxc lxc \</div><div>        params use_screen=true \</div><div>        op stop timeout=300s interval=0</div><div>primitive fs_first @fs \</div><div>        params device="/dev/rbd/rbd/first" fstype=xfs directory="/srv/first" \</div><div>        meta target-role=Stopped</div><div>primitive fs_fs1 @fs \</div><div>        params device="/dev/rbd/rbd/fs1" fstype=xfs directory="/var/lib/lxc/fs1"</div><div>primitive fs_ld1 @fs \</div><div>        params device="/dev/rbd/rbd/ld1" fstype=xfs directory="/var/lib/lxc/ld1"</div><div>primitive fs_ld2 @fs \</div><div>        params device="/dev/rbd/rbd/ld2" fstype=xfs directory="/var/lib/lxc/ld2"</div><div>primitive fs_ld3 @fs \</div><div>        params device="/dev/rbd/rbd/ld3" fstype=xfs directory="/var/lib/lxc/ld3"</div><div>primitive fs_ls1 @fs \</div><div>        params device="/dev/rbd/rbd/ls1" fstype=xfs directory="/var/lib/lxc/ls1"</div><div>primitive fs_ls2 @fs \</div><div>        params device="/dev/rbd/rbd/ls2" fstype=xfs directory="/var/lib/lxc/ls2"</div><div>primitive fs_ls3 @fs \</div><div>        params device="/dev/rbd/rbd/ls3" fstype=xfs directory="/var/lib/lxc/ls3"</div><div>primitive fs_ls4 @fs \</div><div>        params device="/dev/rbd/rbd/ls4" fstype=xfs directory="/var/lib/lxc/ls4"</div><div>primitive fs_ls5 @fs \</div><div>        params device="/dev/rbd/rbd/ls5" fstype=xfs directory="/var/lib/lxc/ls5"</div><div>primitive fs_rdosrv @fs \</div><div>        params device="/dev/rbd/rbd/rdosrv" fstype=xfs directory="/var/lib/lxc/<wbr>rdosrv"</div></div><div><div>primitive fs_unifi @fs \</div><div>        params device="/dev/rbd/rbd/unifi" fstype=xfs directory="/var/lib/lxc/unifi" \</div><div>        meta target-role=Stopped is-managed=false</div><div>primitive kvm_foxy @kvm \</div><div>        params config="/srv/cluster/<wbr>containers/foxy.xml" \</div><div>        utilization cpu=8 hv_memory=4000 \</div><div>        meta target-role=Stopped is-managed=true</div><div>primitive kvm_ws5 @kvm \</div><div>        params config="/srv/cluster/<wbr>containers/ws5.xml" \</div><div>        utilization cpu=4 hv_memory=4096 \</div><div>        meta target-role=Started</div><div>primitive lxc_fs1 @lxc \</div><div>        params container=fs1 config="/var/lib/lxc/fs1/<wbr>config"</div><div>primitive lxc_ld1 @lxc \</div><div>        params container=ld1 config="/var/lib/lxc/ld1/<wbr>config" \</div><div>        meta target-role=Stopped</div><div>primitive lxc_ld2 @lxc \</div><div>        params container=ld2 config="/var/lib/lxc/ld2/<wbr>config"</div><div>primitive lxc_ld3 @lxc \</div><div>        params container=ld3 config="/var/lib/lxc/ld3/<wbr>config"</div><div>primitive lxc_ls1 @lxc \</div><div>        params container=ls1 config="/var/lib/lxc/ls1/<wbr>config"</div><div>primitive lxc_ls2 @lxc \</div><div>        params container=ls2 config="/var/lib/lxc/ls2/<wbr>config"</div><div>primitive lxc_ls3 @lxc \</div><div>        params container=ls3 config="/var/lib/lxc/ls3/<wbr>config"</div><div>primitive lxc_ls4 @lxc \</div><div>        params container=ls4 config="/var/lib/lxc/ls4/<wbr>config"</div></div><div><div>primitive lxc_ls5 @lxc \</div><div>        params container=ls5 config="/var/lib/lxc/ls5/<wbr>config" \</div><div>        meta target-role=Started</div><div>primitive lxc_rdosrv @lxc \</div><div>        params container=rdosrv config="/var/lib/lxc/rdosrv/<wbr>config"</div><div>primitive lxc_unifi @lxc \</div><div>        params container=unifi config="/var/lib/lxc/unifi/<wbr>config" \</div><div>        meta target-role=Stopped is-managed=false</div><div>primitive rbd_first ocf:ceph:rbd \</div><div>        params name=first \</div><div>        meta target-role=Stopped</div><div>primitive rbd_fs1 ocf:ceph:rbd \</div><div>        params name=fs1</div><div>primitive rbd_ld1 ocf:ceph:rbd \</div><div>        params name=ld1</div><div>primitive rbd_ld2 ocf:ceph:rbd \</div><div>        params name=ld2</div><div>primitive rbd_ld3 ocf:ceph:rbd \</div><div>        params name=ld3</div><div>primitive rbd_ls1 ocf:ceph:rbd \</div><div>        params name=ls1</div><div>primitive rbd_ls2 ocf:ceph:rbd \</div><div>        params name=ls2</div><div>primitive rbd_ls3 ocf:ceph:rbd \</div><div>        params name=ls3</div><div>primitive rbd_ls4 ocf:ceph:rbd \</div><div>        params name=ls4</div><div>primitive rbd_ls5 ocf:ceph:rbd \</div><div>        params name=ls5</div><div>primitive rbd_rdosrv ocf:ceph:rbd \</div><div>        params name=rdosrv \</div><div>        meta target-role=Started</div><div>primitive rbd_unifi ocf:ceph:rbd \</div><div>        params name=unifi \</div><div>        meta target-role=Started</div><div>group grp_first rbd_first fs_first \</div><div>        meta is-managed=true</div><div>group grp_fs1 rbd_fs1 fs_fs1 lxc_fs1 \</div><div>        meta is-managed=true target-role=Started</div><div>group grp_ld1 rbd_ld1 fs_ld1 lxc_ld1 \</div><div>        meta target-role=Stopped</div></div><div><div>group grp_ld2 rbd_ld2 fs_ld2 lxc_ld2 \</div><div>        meta target-role=Stopped</div><div>group grp_ld3 rbd_ld3 fs_ld3 lxc_ld3 \</div><div>        meta target-role=Started</div><div>group grp_ls1 rbd_ls1 fs_ls1 lxc_ls1 \</div><div>        meta target-role=Started</div><div>group grp_ls2 rbd_ls2 fs_ls2 lxc_ls2 \</div><div>        meta is-managed=true</div><div>group grp_ls3 rbd_ls3 fs_ls3 lxc_ls3 \</div><div>        meta target-role=Started</div><div>group grp_ls4 rbd_ls4 fs_ls4 lxc_ls4 \</div><div>        meta target-role=Stopped</div><div>group grp_ls5 rbd_ls5 fs_ls5 lxc_ls5 \</div><div>        meta target-role=Started</div><div>group grp_rdosrv rbd_rdosrv fs_rdosrv lxc_rdosrv \</div><div>        meta target-role=Started is-managed=true</div><div>group grp_unifi rbd_unifi fs_unifi lxc_unifi \</div><div>        meta target-role=Started</div><div>location cli-prefer-fs_ls5 fs_ls5 role=Started inf: h3</div><div>location cli-prefer-grp_fs1 grp_fs1 role=Started inf: h4</div><div>location cli-prefer-grp_ld1 grp_ld1 role=Started inf: h3</div><div>location cli-prefer-grp_ld2 grp_ld2 role=Started inf: h2</div><div>location cli-prefer-grp_ld3 grp_ld3 role=Started inf: h4</div><div>location cli-prefer-grp_ls1 grp_ls1 role=Started inf: h2</div><div>location cli-prefer-grp_ls2 grp_ls2 role=Started inf: h2</div><div>location cli-prefer-grp_ls3 grp_ls3 role=Started inf: h4</div><div>location cli-prefer-grp_ls4 grp_ls4 role=Started inf: h3</div><div>location cli-prefer-grp_ls5 grp_ls5 role=Started inf: h4</div><div>location cli-prefer-grp_rdosrv grp_rdosrv role=Started inf: h4</div><div>location cli-prefer-grp_unifi grp_unifi role=Started inf: h4</div><div>location cli-prefer-kvm_foxy kvm_foxy role=Started inf: h1</div><div>location cli-prefer-kvm_ws5 kvm_ws5 role=Started inf: h1</div><div>location cli-prefer-lxc_fs1 lxc_fs1 role=Started inf: h3</div><div>location cli-prefer-lxc_ld3 lxc_ld3 role=Started inf: h3</div><div>location cli-prefer-lxc_ls1 lxc_ls1 role=Started inf: h2</div><div>location cli-prefer-lxc_ls3 lxc_ls3 role=Started inf: h2</div><div>location cli-prefer-lxc_ls4 lxc_ls4 role=Started inf: h3</div><div>location cli-prefer-lxc_ls5 lxc_ls5 role=Started inf: h3</div><div>location cli-prefer-lxc_rdosrv lxc_rdosrv role=Started inf: h3</div><div>location cli-prefer-lxc_unifi lxc_unifi role=Started inf: h3</div><div>location cli-prefer-rbd_ls5 rbd_ls5 role=Started inf: h3</div><div>location cli-prefer-rbd_rdosrv rbd_rdosrv role=Started inf: h4</div></div><div><div>property cib-bootstrap-options: \</div><div>        have-watchdog=false \</div><div>        dc-version=1.1.13-20.1-6f22ad7 \</div><div>        cluster-infrastructure=<wbr>corosync \</div><div>        cluster-name=hosteria \</div><div>        stonith-enabled=false \</div><div>        last-lrm-refresh=1478629707</div><div>rsc_defaults rsc-options: \</div><div>        resource-stickiness=100</div></div><div><br></div><div>Pacemaker logs:</div><div><br></div><div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Forwarding cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='rbd_unifi'] to master (origin=local/crmd/393)</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Diff: --- 0.1445.14 2</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Diff: +++ 0.1445.15 0baf07ee77ccf8d61824f54c68f116<wbr>b7</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>-- /cib/status/node_state[@id='<wbr>167838209']/lrm[@id='<wbr>167838209']/lrm_resources/lrm_<wbr>resource[@id='rbd_unifi']</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=15</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Completed cib_delete operation for section //node_state[@uname='h1']//<wbr>lrm_resource[@id='rbd_unifi']: OK (rc=0, origin=h1/crmd/345, version=0.1445.14)</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Diff: --- 0.1445.14 2</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Diff: +++ 0.1445.15 (null)</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>-- /cib/status/node_state[@id='<wbr>167838212']/lrm[@id='<wbr>167838212']/lrm_resources/lrm_<wbr>resource[@id='rbd_unifi']</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=15</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Completed cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='rbd_unifi']: OK (rc=0, origin=h4/crmd/393, version=0.1445.14)</div><div>Nov 08 09:35:00 [3567] h4       crmd:     info: delete_resource:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Removing resource rbd_unifi for d44ab7d6-13f8-4a8c-a9f9-<wbr>ffeeda6727a2 (root) on h4</div><div>Nov 08 09:35:00 [3567] h4       crmd:     info: notify_deleted:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Notifying d44ab7d6-13f8-4a8c-a9f9-<wbr>ffeeda6727a2 on h4 that rbd_unifi was deleted</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Forwarding cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='rbd_unifi'] to master (origin=local/crmd/394)</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Forwarding cib_modify operation for section crm_config to master (origin=local/crmd/396)</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Diff: --- 0.1445.14 2</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Diff: +++ 0.1445.15 (null)</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>-- /cib/status/node_state[@id='<wbr>167838212']/lrm[@id='<wbr>167838212']/lrm_resources/lrm_<wbr>resource[@id='rbd_unifi']</div><div>Nov 08 09:35:00 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=15</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Completed cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='rbd_unifi']: OK (rc=0, origin=h4/crmd/394, version=0.1445.15)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Diff: --- 0.1445.15 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Diff: +++ 0.1446.0 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>+  /cib:  @epoch=1446, @num_updates=0</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>+  /cib/configuration/crm_<wbr>config/cluster_property_set[@<wbr>id='cib-bootstrap-options']/<wbr>nvpair[@id='cib-bootstrap-<wbr>options-last-lrm-refresh']:  @value=1478626500</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h4/crmd/396, version=0.1446.0)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Forwarding cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='fs_unifi'] to master (origin=local/crmd/397)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Diff: --- 0.1446.0 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1446.1 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838212']/lrm[@id='<wbr>167838212']/lrm_resources/lrm_<wbr>resource[@id='fs_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>+  /cib:  @num_updates=1</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='fs_unifi']: OK (rc=0, origin=h4/crmd/397, version=0.1446.0)</div><div>Nov 08 09:35:01 [3567] h4       crmd:     info: delete_resource:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Removing resource fs_unifi for d44ab7d6-13f8-4a8c-a9f9-<wbr>ffeeda6727a2 (root) on h4</div><div>Nov 08 09:35:01 [3567] h4       crmd:     info: notify_deleted:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>Notifying d44ab7d6-13f8-4a8c-a9f9-<wbr>ffeeda6727a2 on h4 that fs_unifi was deleted</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Diff: --- 0.1446.0 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1446.1 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838209']/lrm[@id='<wbr>167838209']/lrm_resources/lrm_<wbr>resource[@id='rbd_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=1</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_file_backup:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Archived previous version as /var/lib/pacemaker/cib/cib-64.<wbr>raw</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Completed cib_delete operation for section //node_state[@uname='h1']//<wbr>lrm_resource[@id='rbd_unifi']: OK (rc=0, origin=h1/crmd/346, version=0.1446.1)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Diff: --- 0.1446.1 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.0 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>+  /cib:  @epoch=1447, @num_updates=0</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>+  /cib/configuration/crm_<wbr>config/cluster_property_set[@<wbr>id='cib-bootstrap-options']/<wbr>nvpair[@id='cib-bootstrap-<wbr>options-last-lrm-refresh']:  @value=1478626267</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h1/crmd/348, version=0.1447.0)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1447.0 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.1 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838209']/lrm[@id='<wbr>167838209']/lrm_resources/lrm_<wbr>resource[@id='fs_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>+  /cib:  @num_updates=1</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h1']//<wbr>lrm_resource[@id='fs_unifi']: OK (rc=0, origin=h1/crmd/349, version=0.1447.0)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1447.0 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.1 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838209']/lrm[@id='<wbr>167838209']/lrm_resources/lrm_<wbr>resource[@id='fs_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>+  /cib:  @num_updates=1</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h1']//<wbr>lrm_resource[@id='fs_unifi']: OK (rc=0, origin=h1/crmd/350, version=0.1447.1)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_file_write_with_digest:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>Wrote version 0.1446.0 of the CIB to disk (digest: 13247e2b093b5d2dee902604bc9a32<wbr>08)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h1/crmd/352, version=0.1447.1)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_file_write_with_digest:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>Reading cluster configuration file /var/lib/pacemaker/cib/cib.<wbr>111AJI (digest: /var/lib/pacemaker/cib/cib.<wbr>Zv2YLg)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1447.1 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.2 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838209']/lrm[@id='<wbr>167838209']/lrm_resources/lrm_<wbr>resource[@id='lxc_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h1']//<wbr>lrm_resource[@id='lxc_unifi']: OK (rc=0, origin=h1/crmd/353, version=0.1447.1)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Diff: --- 0.1447.1 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.2 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838209']/lrm[@id='<wbr>167838209']/lrm_resources/lrm_<wbr>resource[@id='lxc_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h1']//<wbr>lrm_resource[@id='lxc_unifi']: OK (rc=0, origin=h1/crmd/354, version=0.1447.2)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h1/crmd/356, version=0.1447.2)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1447.2 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.3 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838210']/lrm[@id='<wbr>167838210']/lrm_resources/lrm_<wbr>resource[@id='rbd_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=3</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h2']//<wbr>lrm_resource[@id='rbd_unifi']: OK (rc=0, origin=h2/crmd/346, version=0.1447.2)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Forwarding cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='fs_unifi'] to master (origin=local/crmd/398)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Diff: --- 0.1447.2 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.3 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838212']/lrm[@id='<wbr>167838212']/lrm_resources/lrm_<wbr>resource[@id='fs_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>+  /cib:  @num_updates=3</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='fs_unifi']: OK (rc=0, origin=h4/crmd/398, version=0.1447.3)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1447.3 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.4 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838210']/lrm[@id='<wbr>167838210']/lrm_resources/lrm_<wbr>resource[@id='rbd_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=4</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h2']//<wbr>lrm_resource[@id='rbd_unifi']: OK (rc=0, origin=h2/crmd/347, version=0.1447.4)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h2/crmd/349, version=0.1447.4)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1447.4 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.5 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838210']/lrm[@id='<wbr>167838210']/lrm_resources/lrm_<wbr>resource[@id='fs_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>+  /cib:  @num_updates=5</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h2']//<wbr>lrm_resource[@id='fs_unifi']: OK (rc=0, origin=h2/crmd/350, version=0.1447.4)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1447.4 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.5 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838210']/lrm[@id='<wbr>167838210']/lrm_resources/lrm_<wbr>resource[@id='fs_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>+  /cib:  @num_updates=5</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h2']//<wbr>lrm_resource[@id='fs_unifi']: OK (rc=0, origin=h2/crmd/351, version=0.1447.5)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h2/crmd/353, version=0.1447.5)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1447.5 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.6 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838210']/lrm[@id='<wbr>167838210']/lrm_resources/lrm_<wbr>resource[@id='lxc_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=6</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h2']//<wbr>lrm_resource[@id='lxc_unifi']: OK (rc=0, origin=h2/crmd/354, version=0.1447.5)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Diff: --- 0.1447.5 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1447.6 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838210']/lrm[@id='<wbr>167838210']/lrm_resources/lrm_<wbr>resource[@id='lxc_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=6</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h2']//<wbr>lrm_resource[@id='lxc_unifi']: OK (rc=0, origin=h2/crmd/355, version=0.1447.6)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h2/crmd/357, version=0.1447.6)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Forwarding cib_modify operation for section crm_config to master (origin=local/crmd/400)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Forwarding cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='lxc_unifi'] to master (origin=local/crmd/401)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Diff: --- 0.1447.6 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1448.0 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>+  /cib:  @epoch=1448, @num_updates=0</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>+  /cib/configuration/crm_<wbr>config/cluster_property_set[@<wbr>id='cib-bootstrap-options']/<wbr>nvpair[@id='cib-bootstrap-<wbr>options-last-lrm-refresh']:  @value=1478626501</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h4/crmd/400, version=0.1448.0)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1448.0 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1448.1 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838212']/lrm[@id='<wbr>167838212']/lrm_resources/lrm_<wbr>resource[@id='lxc_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=1</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='lxc_unifi']: OK (rc=0, origin=h4/crmd/401, version=0.1448.0)</div><div>Nov 08 09:35:01 [3567] h4       crmd:     info: delete_resource:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Removing resource lxc_unifi for d44ab7d6-13f8-4a8c-a9f9-<wbr>ffeeda6727a2 (root) on h4</div><div>Nov 08 09:35:01 [3567] h4       crmd:     info: notify_deleted:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Notifying d44ab7d6-13f8-4a8c-a9f9-<wbr>ffeeda6727a2 on h4 that lxc_unifi was deleted</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Forwarding cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='lxc_unifi'] to master (origin=local/crmd/402)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Forwarding cib_modify operation for section crm_config to master (origin=local/crmd/404)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Diff: --- 0.1448.0 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1448.1 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838212']/lrm[@id='<wbr>167838212']/lrm_resources/lrm_<wbr>resource[@id='lxc_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=1</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h4']//<wbr>lrm_resource[@id='lxc_unifi']: OK (rc=0, origin=h4/crmd/402, version=0.1448.1)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h4/crmd/404, version=0.1448.1)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1448.1 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1448.2 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838211']/lrm[@id='<wbr>167838211']/lrm_resources/lrm_<wbr>resource[@id='rbd_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h3']//<wbr>lrm_resource[@id='rbd_unifi']: OK (rc=0, origin=h3/crmd/771, version=0.1448.1)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Diff: --- 0.1448.1 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1448.2 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838211']/lrm[@id='<wbr>167838211']/lrm_resources/lrm_<wbr>resource[@id='rbd_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h3']//<wbr>lrm_resource[@id='rbd_unifi']: OK (rc=0, origin=h3/crmd/772, version=0.1448.2)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Diff: --- 0.1448.2 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.0 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>+  /cib:  @epoch=1449, @num_updates=0</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>+  /cib/configuration/crm_<wbr>config/cluster_property_set[@<wbr>id='cib-bootstrap-options']/<wbr>nvpair[@id='cib-bootstrap-<wbr>options-last-lrm-refresh']:  @value=1478626267</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h3/crmd/774, version=0.1449.0)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_file_backup:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Archived previous version as /var/lib/pacemaker/cib/cib-65.<wbr>raw</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_file_write_with_digest:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Wrote version 0.1449.0 of the CIB to disk (digest: f6be1cc446b06cf40d434f709a1e66<wbr>0e)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_file_write_with_digest:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Reading cluster configuration file /var/lib/pacemaker/cib/cib.<wbr>dq077Z (digest: /var/lib/pacemaker/cib/cib.<wbr>HTyLSy)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1449.0 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.1 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838211']/lrm[@id='<wbr>167838211']/lrm_resources/lrm_<wbr>resource[@id='fs_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>+  /cib:  @num_updates=1</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h3']//<wbr>lrm_resource[@id='fs_unifi']: OK (rc=0, origin=h3/crmd/789, version=0.1449.0)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1449.0 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.1 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838211']/lrm[@id='<wbr>167838211']/lrm_resources/lrm_<wbr>resource[@id='fs_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>+  /cib:  @num_updates=1</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h3']//<wbr>lrm_resource[@id='fs_unifi']: OK (rc=0, origin=h3/crmd/790, version=0.1449.1)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h3/crmd/792, version=0.1449.1)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1449.1 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.2 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838211']/lrm[@id='<wbr>167838211']/lrm_resources/lrm_<wbr>resource[@id='lxc_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h3']//<wbr>lrm_resource[@id='lxc_unifi']: OK (rc=0, origin=h3/crmd/796, version=0.1449.1)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Diff: --- 0.1449.1 2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.2 (null)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>-- /cib/status/node_state[@id='<wbr>167838211']/lrm[@id='<wbr>167838211']/lrm_resources/lrm_<wbr>resource[@id='lxc_unifi']</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>+  /cib:  @num_updates=2</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Completed cib_delete operation for section //node_state[@uname='h3']//<wbr>lrm_resource[@id='lxc_unifi']: OK (rc=0, origin=h3/crmd/797, version=0.1449.2)</div><div>Nov 08 09:35:01 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Completed cib_modify operation for section crm_config: OK (rc=0, origin=h3/crmd/799, version=0.1449.2)</div><div>Nov 08 09:35:05 [3564] h4       lrmd:     info: process_lrmd_get_rsc_info:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Resource 'rbd_unifi' not found (34 active resources)</div><div>Nov 08 09:35:05 [3564] h4       lrmd:     info: process_lrmd_rsc_register:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Added 'rbd_unifi' to the rsc list (35 active resources)</div><div>Nov 08 09:35:05 [3567] h4       crmd:     info: do_lrm_rsc_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Performing key=12:121:7:2bc5b523-b947-<wbr>4035-b296-4c099646298b op=rbd_unifi_monitor_0</div><div>Nov 08 09:35:05 [3564] h4       lrmd:     info: process_lrmd_get_rsc_info:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Resource 'fs_unifi' not found (35 active resources)</div><div>Nov 08 09:35:05 [3564] h4       lrmd:     info: process_lrmd_rsc_register:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Added 'fs_unifi' to the rsc list (36 active resources)</div><div>Nov 08 09:35:05 [3567] h4       crmd:     info: do_lrm_rsc_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Performing key=13:121:7:2bc5b523-b947-<wbr>4035-b296-4c099646298b op=fs_unifi_monitor_0</div><div>Nov 08 09:35:05 [3564] h4       lrmd:     info: process_lrmd_get_rsc_info:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Resource 'lxc_unifi' not found (36 active resources)</div><div>Nov 08 09:35:05 [3564] h4       lrmd:     info: process_lrmd_rsc_register:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Added 'lxc_unifi' to the rsc list (37 active resources)</div><div>Nov 08 09:35:05 [3567] h4       crmd:     info: do_lrm_rsc_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Performing key=14:121:7:2bc5b523-b947-<wbr>4035-b296-4c099646298b op=lxc_unifi_monitor_0</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">        </span>Diff: --- 0.1449.2 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.3 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>+  /cib:  @num_updates=3</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>++ /cib/status/node_state[@id='<wbr>167838209']/lrm[@id='<wbr>167838209']/lrm_resources:  <lrm_resource id="rbd_unifi" type="rbd" class="ocf" provider="ceph"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>++                                                                                <lrm_rsc_op id="rbd_unifi_last_0" operation_key="rbd_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="3:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:7;3:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h1" call-id="441" rc-code="7" op-status="0" interval="0" last-run="147862</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h1/crmd/357, version=0.1449.3)</div><div>Nov 08 09:35:05 [3564] h4       lrmd:   notice: operation_finished:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>lxc_unifi_monitor_0:23285:<wbr>stderr [ unifi doesn't exist ]</div><div>Nov 08 09:35:05 [3567] h4       crmd:   notice: process_lrm_event:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Operation lxc_unifi_monitor_0: not running (node=h4, call=499, rc=7, cib-update=405, confirmed=true)</div><div>Nov 08 09:35:05 [3567] h4       crmd:   notice: process_lrm_event:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>h4-lxc_unifi_monitor_0:499 [ unifi doesn't exist\n ]</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Diff: --- 0.1449.3 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.4 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>+  /cib:  @num_updates=4</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>++ /cib/status/node_state[@id='<wbr>167838210']/lrm[@id='<wbr>167838210']/lrm_resources:  <lrm_resource id="lxc_unifi" type="lxc" class="ocf" provider="heartbeat"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>++                                                                                <lrm_rsc_op id="lxc_unifi_last_0" operation_key="lxc_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="8:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:7;8:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h2" call-id="451" rc-code="7" op-status="0" interval="0" last-run="147862</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h2/crmd/358, version=0.1449.4)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Forwarding cib_modify operation for section status to master (origin=local/crmd/405)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: --- 0.1449.4 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.5 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>+  /cib:  @num_updates=5</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>++ /cib/status/node_state[@id='<wbr>167838212']/lrm[@id='<wbr>167838212']/lrm_resources:  <lrm_resource id="lxc_unifi" type="lxc" class="ocf" provider="heartbeat"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>++                                                                                <lrm_rsc_op id="lxc_unifi_last_0" operation_key="lxc_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="14:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:7;14:121:<wbr>7:2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h4" call-id="499" rc-code="7" op-status="0" interval="0" last-run="1478</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h4/crmd/405, version=0.1449.5)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>Diff: --- 0.1449.5 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.6 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>+  /cib:  @num_updates=6</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>++ /cib/status/node_state[@id='<wbr>167838210']/lrm[@id='<wbr>167838210']/lrm_resources:  <lrm_resource id="fs_unifi" type="Filesystem" class="ocf" provider="heartbeat"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>++                                                                                <lrm_rsc_op id="fs_unifi_last_0" operation_key="fs_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="7:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:7;7:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h2" call-id="447" rc-code="7" op-status="0" interval="0" last-run="14786262</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h2/crmd/359, version=0.1449.6)</div><div>Nov 08 09:35:05 [3567] h4       crmd:   notice: process_lrm_event:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Operation fs_unifi_monitor_0: not running (node=h4, call=495, rc=7, cib-update=406, confirmed=true)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Diff: --- 0.1449.6 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.7 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>+  /cib:  @num_updates=7</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>++ /cib/status/node_state[@id='<wbr>167838209']/lrm[@id='<wbr>167838209']/lrm_resources:  <lrm_resource id="lxc_unifi" type="lxc" class="ocf" provider="heartbeat"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>++                                                                                <lrm_rsc_op id="lxc_unifi_last_0" operation_key="lxc_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="5:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:7;5:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h1" call-id="449" rc-code="7" op-status="0" interval="0" last-run="147862</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h1/crmd/358, version=0.1449.7)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>Forwarding cib_modify operation for section status to master (origin=local/crmd/406)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: --- 0.1449.7 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.8 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>+  /cib:  @num_updates=8</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>++ /cib/status/node_state[@id='<wbr>167838212']/lrm[@id='<wbr>167838212']/lrm_resources:  <lrm_resource id="fs_unifi" type="Filesystem" class="ocf" provider="heartbeat"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>++                                                                                <lrm_rsc_op id="fs_unifi_last_0" operation_key="fs_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="13:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:7;13:121:<wbr>7:2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h4" call-id="495" rc-code="7" op-status="0" interval="0" last-run="147862</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h4/crmd/406, version=0.1449.8)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>Diff: --- 0.1449.8 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.9 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>+  /cib:  @num_updates=9</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>++ /cib/status/node_state[@id='<wbr>167838211']/lrm[@id='<wbr>167838211']/lrm_resources:  <lrm_resource id="lxc_unifi" type="lxc" class="ocf" provider="heartbeat"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">       </span>++                                                                                <lrm_rsc_op id="lxc_unifi_last_failure_0" operation_key="lxc_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="11:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:0;11:121:<wbr>7:2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h3" call-id="465" rc-code="0" op-status="0" interval="0" last-r</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>++                                                                                <lrm_rsc_op id="lxc_unifi_last_0" operation_key="lxc_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="11:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:0;11:121:<wbr>7:2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h3" call-id="465" rc-code="0" op-status="0" interval="0" last-run="1478</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h3/crmd/802, version=0.1449.9)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>Diff: --- 0.1449.9 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: +++ 0.1449.10 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>+  /cib:  @num_updates=10</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>++ /cib/status/node_state[@id='<wbr>167838209']/lrm[@id='<wbr>167838209']/lrm_resources:  <lrm_resource id="fs_unifi" type="Filesystem" class="ocf" provider="heartbeat"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>++                                                                                <lrm_rsc_op id="fs_unifi_last_0" operation_key="fs_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="4:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:7;4:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h1" call-id="445" rc-code="7" op-status="0" interval="0" last-run="14786262</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h1/crmd/359, version=0.1449.10)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Diff: --- 0.1449.10 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Diff: +++ 0.1449.11 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>+  /cib:  @num_updates=11</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>++ /cib/status/node_state[@id='<wbr>167838211']/lrm[@id='<wbr>167838211']/lrm_resources:  <lrm_resource id="fs_unifi" type="Filesystem" class="ocf" provider="heartbeat"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>++                                                                                <lrm_rsc_op id="fs_unifi_last_failure_0" operation_key="fs_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="10:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:0;10:121:<wbr>7:2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h3" call-id="461" rc-code="0" op-status="0" interval="0" last-run</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>++                                                                                <lrm_rsc_op id="fs_unifi_last_0" operation_key="fs_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="10:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:0;10:121:<wbr>7:2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h3" call-id="461" rc-code="0" op-status="0" interval="0" last-run="147862</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h3/crmd/803, version=0.1449.11)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>Diff: --- 0.1449.11 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Diff: +++ 0.1449.12 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>+  /cib:  @num_updates=12</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>++ /cib/status/node_state[@id='<wbr>167838210']/lrm[@id='<wbr>167838210']/lrm_resources:  <lrm_resource id="rbd_unifi" type="rbd" class="ocf" provider="ceph"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>++                                                                                <lrm_rsc_op id="rbd_unifi_last_0" operation_key="rbd_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="6:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:7;6:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h2" call-id="443" rc-code="7" op-status="0" interval="0" last-run="147862</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h2/crmd/360, version=0.1449.12)</div><div>Nov 08 09:35:05 [3567] h4       crmd:   notice: process_lrm_event:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Operation rbd_unifi_monitor_0: not running (node=h4, call=491, rc=7, cib-update=407, confirmed=true)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: --- 0.1449.12 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Diff: +++ 0.1449.13 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>+  /cib:  @num_updates=13</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>++ /cib/status/node_state[@id='<wbr>167838211']/lrm[@id='<wbr>167838211']/lrm_resources:  <lrm_resource id="rbd_unifi" type="rbd" class="ocf" provider="ceph"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>++                                                                                <lrm_rsc_op id="rbd_unifi_last_failure_0" operation_key="rbd_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="9:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:0;9:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h3" call-id="457" rc-code="0" op-status="0" interval="0" last-run</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>++                                                                                <lrm_rsc_op id="rbd_unifi_last_0" operation_key="rbd_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="9:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:0;9:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h3" call-id="457" rc-code="0" op-status="0" interval="0" last-run="147862</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h3/crmd/804, version=0.1449.13)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">      </span>Forwarding cib_modify operation for section status to master (origin=local/crmd/407)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">  </span>Diff: --- 0.1449.13 2</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Diff: +++ 0.1449.14 (null)</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>+  /cib:  @num_updates=14</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">   </span>++ /cib/status/node_state[@id='<wbr>167838212']/lrm[@id='<wbr>167838212']/lrm_resources:  <lrm_resource id="rbd_unifi" type="rbd" class="ocf" provider="ceph"/></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">    </span>++                                                                                <lrm_rsc_op id="rbd_unifi_last_0" operation_key="rbd_unifi_<wbr>monitor_0" operation="monitor" crm-debug-origin="do_update_<wbr>resource" crm_feature_set="3.0.10" transition-key="12:121:7:<wbr>2bc5b523-b947-4035-b296-<wbr>4c099646298b" transition-magic="0:7;12:121:<wbr>7:2bc5b523-b947-4035-b296-<wbr>4c099646298b" on_node="h4" call-id="491" rc-code="7" op-status="0" interval="0" last-run="1478</div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_perform_op:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>++                                                                              </lrm_resource></div><div>Nov 08 09:35:05 [3562] h4        cib:     info: cib_process_request:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap">     </span>Completed cib_modify operation for section status: OK (rc=0, origin=h4/crmd/407, version=0.1449.14)</div><div>Nov 08 09:35:10 [3562] h4        cib:     info: cib_process_ping:<span class="m_7866109381349217603Apple-tab-span" style="white-space:pre-wrap"> </span>Reporting our current digest to h3: b5a63d2ac8b4835f8c8a2f310eb664<wbr>8c for 0.1449.14 (0x12ddc90 0)</div></div><br></div></div></div>
<br>______________________________<wbr>_________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org">Users@clusterlabs.org</a><br>
<a href="http://clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://clusterlabs.org/<wbr>mailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">http://www.clusterlabs.org/<wbr>doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
<br></blockquote></div><br></div></div>