[ClusterLabs] packmaker: After migrate a resource, changing resource non-unique param lead to resoruce restart
Ken Gaillot
kgaillot at redhat.com
Thu Nov 3 15:53:24 UTC 2016
On 11/03/2016 02:01 AM, 李清硕 wrote:
> Hi everyone,
> I'm testing pacemaker resoruce live migration, in a simple test
> environment with two virtual machines.
> The resource is something encapsulated kvm, when i perfrom migrate, for
> example, form node1 to node2,
> i notice, the pacemaker invoke migrate_to, a one-off monitor on node1,
> like this:
> (migrate_to:rgavhcnP-iOgMD4-Bdht) ACTION BEDIN
> (migrate_to:rgavhcnP-iOgMD4-Bdht) ACTION EDN WITH 0
> (monitor:rgavhcnP-iOgMD4-Bdht) ACTION BEDIN
> (monitor:rgavhcnP-iOgMD4-Bdht) ACTION EDN WITH 7
> invoke two one-off monitor on node2 like this:
> (monitor:rgavhcnP-iOgMD4-Bdht) ACTION BEDIN
> (monitor:rgavhcnP-iOgMD4-Bdht) ACTION EDN WITH 0
> (monitor:rgavhcnP-iOgMD4-Bdht) ACTION BEDIN
> (monitor:rgavhcnP-iOgMD4-Bdht) ACTION EDN WITH 0
> When i perfrom crm resource operations resoruce_name(actually i don't
> what this is, maybe lead to the resource restart) , the output is:
> rgavhcnP-iOgMD4-Bdh (ocf::heartbeat:fronvm) Started:
> rgavhcnP-iOgMD4-Bdht_monitor_0 (node=h122, call=41, rc=0,
> last-rc-change=Thu Nov 3 14:33:25 2016, exec=232ms): complete
> rgavhcnP-iOgMD4-Bdht (ocf::heartbeat:fronvm): Started:
> rgavhcnP-iOgMD4-Bdht_monitor_0 (node=h122, call=41, rc=0,
> last-rc-change=Thu Nov 3 14:33:25 2016, exec=232ms): complete
>
> rgavhcnP-iOgMD4-Bdht (ocf::heartbeat:fronvm): Started:rgavhcnP-iOgMD4-Bdht_monitor_120000
> (node=h122, call=42, rc=0, last-rc-change=Thu Nov 3 14:33:34
> 2016,exec=197ms):complete
> rgavhcnP-iOgMD4-Bdht (ocf::heartbeat:fronvm): Started:
> rgavhcnP-iOgMD4-Bdht_monitor_0 (node=h123, call=35, rc=7,
> last-rc-change=Thu Nov 3 14:34:25 2016, exec=213ms): complete
> The last i change resource params "config" (config is a param name),
> would lead to resource restart, which i expect reload, not restart.
> Is that a normal or a bug or i do something wrong. if you need more
> detail, please tell me, Any hint will help
Pacemaker should reload a resource after a parameter change if:
* unique=0 in the parameter's metadata in the resource agent (<parameter
name=... unique="0">)
* the resource agent advertises support for the reload operation in its
metadata (<action name="reload" ... />)
* no unique parameter was also changed at the same time
Are all those true in this case?
> Thank you！
>
> The pacemaker config:
> node 1084759930: h122
> node 1084759931: h123
> primitive fence stonith:fence_agent \
> params pcmk_host_check=none action=reboot \
> op monitor interval=3600s \
> meta target-role=Started
> primitive rgavhcnP-iOgMD4-Bdht fronvm \
> params config="/mnt/m6dd63be9650f737da8c65d6caa00bc00"
> vm_name=rgavhcnP-iOgMD4-Bdht \
> op start timeout=600s interval=0 \
> op stop timeout=180s interval=0 \
> op migrate_to timeout=1800s interval=0 \
> op migrate_from timeout=60s interval=0 \
> op monitor interval=120s timeout=60s on-fail=ignore \
> meta target-role=Started allow-migrate=true
> clone fence-clone fence \
> meta target-role=Started
> location loc-fence-h122 fence-clone 100: h122
> location loc-fence-h123 fence-clone 100: h123
> location loc-paPupXaC-Ns5mnb-Xdfe-rgavhcnP-iOgMD4-Bdht
> rgavhcnP-iOgMD4-Bdht 1: h122
> location loc-tEJVJGWl-ifcTSp-Ta1v-rgavhcnP-iOgMD4-Bdht
> rgavhcnP-iOgMD4-Bdht 2: h123
> property cib-bootstrap-options: \
> stonith-enabled=true \
> stonith-action=reboot \
> stonith-timeout=120s \
> no-quorum-policy=stop \
> symmetric-cluster=false \
> crmd-transition-delay=5s \
> start-failure-is-fatal=FALSE \
> have-watchdog=false \
> dc-version=1.1.13-10.el7.centos-44eb2dd \
> cluster-infrastructure=corosync \
> cluster-name=OQSQoxu5XrsUcds8 \
> last-lrm-refresh=1478154860
> rsc_defaults rsc-options: \
> resource-stickiness=1024 \
> migration-threshold=3
>
> The log:
> Nov 03 14:34:29 [87769] h123 cib: ( cib_file.c:293 ) info:
> cib_file_backup:Archived previous version as
> /var/lib/pacemaker/cib/cib-83.raw
> Nov 03 14:34:29 [87769] h123 cib: ( cib_file.c:423 ) info:
> cib_file_write_with_digest:Wrote version 0.494.0 of the CIB to disk
> (digest: 97a2f450d62400802a2e04cf20ae010c)
> Nov 03 14:34:29 [87769] h123 cib: ( cib_file.c:442 ) info:
> cib_file_write_with_digest:Reading cluster configuration file
> /var/lib/pacemaker/cib/cib.lTaDlA (digest:
> /var/lib/pacemaker/cib/cib.IlPpJD)
> Nov 03 14:34:34 [87769] h123 cib: ( messages.c:239 ) info:
> cib_process_ping:Reporting our current digest to h122:
> 01ebcac1c89e4493808c256c6c4b86a4 for 0.494.0 (0x2265720 0)
> Nov 03 14:34:34 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.494.0 2
> Nov 03 14:34:34 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.494.1 (null)
> Nov 03 14:34:34 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=1
> Nov 03 14:34:34 [87769] h123 cib: ( xml.c:1684 ) info:
> cib_perform_op:++
> /cib/status/node_state[@id='1084759930']/lrm[@id='1084759930']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> <lrm_rsc_op id="rgavhcnP-iOgMD4-Bdht_monitor_120000"
> operation_key="rgavhcnP-iOgMD4-Bdht_monitor_120000" operation="monitor"
> crm-debug-origin="do_update_resource" crm_feature_set="3.0.10"
> transition-key="17:11:0:3c5a583b-d35c-4623-810f-2c63304cb99c"
> transition-magic="0:0;17:11:0:3c5a583b-d35c-4623-8
> Nov 03 14:34:34 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section status:
> OK (rc=0, origin=h122/crmd/101, version=0.494.1)
> Nov 03 14:34:39 [87769] h123 cib: ( messages.c:239 ) info:
> cib_process_ping:Reporting our current digest to h122:
> 37488a1871c9e00464c472665ccc0612 for 0.494.1 (0x2265720 0)
> Nov 03 14:51:19 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> constraints: OK (rc=0, origin=h122/crm_resource/3, version=0.494.1)
> Nov 03 14:51:19 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.494.1 2
> Nov 03 14:51:19 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.495.0 (null)
> Nov 03 14:51:19 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @epoch=495, @num_updates=0
> Nov 03 14:51:19 [87769] h123 cib: ( xml.c:1684 ) info:
> cib_perform_op:++ /cib/configuration/constraints: <rsc_location
> id="cli-prefer-rgavhcnP-iOgMD4-Bdht" rsc="rgavhcnP-iOgMD4-Bdht"
> role="Started" node="h123" score="INFINITY"/>
> Nov 03 14:51:19 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section
> constraints: OK (rc=0, origin=h122/crm_resource/4, version=0.495.0)
> Nov 03 14:51:19 [87769] h123 cib: ( cib_file.c:293 ) info:
> cib_file_backup:Archived previous version as
> /var/lib/pacemaker/cib/cib-84.raw
> Nov 03 14:51:19 [87769] h123 cib: ( cib_file.c:423 ) info:
> cib_file_write_with_digest:Wrote version 0.495.0 of the CIB to disk
> (digest: d840a1c677c6b841fd88af71a6cc3b95)
> Nov 03 14:51:19 [87769] h123 cib: ( cib_file.c:442 ) info:
> cib_file_write_with_digest:Reading cluster configuration file
> /var/lib/pacemaker/cib/cib.IkzPcM (digest:
> /var/lib/pacemaker/cib/cib.776nJV)
> Nov 03 14:51:24 [87769] h123 cib: ( messages.c:239 ) info:
> cib_process_ping:Reporting our current digest to h122:
> b730a9973a4c9894617b77cc616d3d50 for 0.495.0 (0x2264010 0)
> Nov 03 14:51:24 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.495.0 2
> Nov 03 14:51:24 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.495.1 (null)
> Nov 03 14:51:24 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=1
> Nov 03 14:51:24 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+
> /cib/status/node_state[@id='1084759930']/lrm[@id='1084759930']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']/lrm_rsc_op[@id='rgavhcnP-iOgMD4-Bdht_last_0']:
> @operation_key=rgavhcnP-iOgMD4-Bdht_migrate_to_0,
> @operation=migrate_to, @crm-debug-origin=cib_action_update,
> @transition-key=19:13:0:3c5a583b-d35c-4623-810f-2c63304cb99c,
> @transition-magic=-1:193;19:13:0:3c5a583b-d35c-4623-810f-2c63304cb99c,
> @call-id=-1
> Nov 03 14:51:24 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section status:
> OK (rc=0, origin=h122/crmd/104, version=0.495.1)
> Nov 03 14:51:29 [87769] h123 cib: ( messages.c:239 ) info:
> cib_process_ping:Reporting our current digest to h122:
> b1cce1592606dfad0d73ce9aa072616d for 0.495.1 (0x2264010 0)
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.495.1 2
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.495.2 3e857146f73cc588f61f7512bd167045
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1735 ) info:
> cib_perform_op:--
> /cib/status/node_state[@id='1084759930']/lrm[@id='1084759930']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=2
> Nov 03 14:52:24 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> //node_state[@uname='h122']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> OK (rc=0, origin=h122/crmd/105, version=0.495.1)
> Nov 03 14:52:24 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_delete operation for section
> //node_state[@uname='h123']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht'] to
> master (origin=local/crmd/41)
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.495.1 2
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.495.2 (null)
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1735 ) info:
> cib_perform_op:--
> /cib/status/node_state[@id='1084759931']/lrm[@id='1084759931']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=2
> Nov 03 14:52:24 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> //node_state[@uname='h123']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> OK (rc=0, origin=h123/crmd/41, version=0.495.1)
> Nov 03 14:52:24 [87774] h123 crmd: ( lrm.c:1272 ) info:
> delete_resource:Removing resource rgavhcnP-iOgMD4-Bdht for
> c2aedfce-e95f-4bcf-a009-c7703b8277ec (root) on h122
> Nov 03 14:52:24 [87774] h123 crmd: ( lrm.c:890 ) info:
> notify_deleted:Notifying c2aedfce-e95f-4bcf-a009-c7703b8277ec on h122
> that rgavhcnP-iOgMD4-Bdht was deleted
> Nov 03 14:52:24 [87770] h123 stonith-ng: ( xml.c:2055 ) info:
> xml_patch_version_check:Current num_updates is too high (2 > 1)
> Nov 03 14:52:24 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_delete operation for section
> //node_state[@uname='h123']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht'] to
> master (origin=local/crmd/42)
> Nov 03 14:52:24 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_modify operation for section
> crm_config to master (origin=local/crmd/44)
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.495.1 2
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.495.2 (null)
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1735 ) info:
> cib_perform_op:--
> /cib/status/node_state[@id='1084759931']/lrm[@id='1084759931']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=2
> Nov 03 14:52:24 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> //node_state[@uname='h123']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> OK (rc=0, origin=h123/crmd/42, version=0.495.2)
> Nov 03 14:52:24 [87770] h123 stonith-ng: ( xml.c:2055 ) info:
> xml_patch_version_check:Current num_updates is too high (2 > 1)
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.495.2 2
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.496.0 (null)
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @epoch=496, @num_updates=0
> Nov 03 14:52:24 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+
> /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-last-lrm-refresh']:
> @value=1478155944
> Nov 03 14:52:24 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section
> crm_config: OK (rc=0, origin=h123/crmd/44, version=0.496.0)
> Nov 03 14:52:24 [87774] h123 crmd: ( notify.c:55 ) notice:
> crmd_enable_notifications:Notifications disabled
> Nov 03 14:52:24 [87769] h123 cib: ( cib_file.c:293 ) info:
> cib_file_backup:Archived previous version as
> /var/lib/pacemaker/cib/cib-85.raw
> Nov 03 14:52:24 [87769] h123 cib: ( cib_file.c:423 ) info:
> cib_file_write_with_digest:Wrote version 0.496.0 of the CIB to disk
> (digest: a7f1f796f7289f9cbfaae2134ad332b4)
> Nov 03 14:52:24 [87769] h123 cib: ( cib_file.c:442 ) info:
> cib_file_write_with_digest:Reading cluster configuration file
> /var/lib/pacemaker/cib/cib.jrqiqR (digest:
> /var/lib/pacemaker/cib/cib.a4JCY0)
> Nov 03 14:52:25 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.496.0 2
> Nov 03 14:52:25 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.496.1 (null)
> Nov 03 14:52:25 [87769] h123 cib: ( xml.c:1735 ) info:
> cib_perform_op:--
> /cib/status/node_state[@id='1084759930']/lrm[@id='1084759930']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']
> Nov 03 14:52:25 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=1
> Nov 03 14:52:25 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> //node_state[@uname='h122']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> OK (rc=0, origin=h122/crmd/107, version=0.496.1)
> Nov 03 14:52:25 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.496.1 2
> Nov 03 14:52:25 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.497.0 (null)
> Nov 03 14:52:25 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @epoch=497, @num_updates=0
> Nov 03 14:52:25 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+
> /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-last-lrm-refresh']:
> @value=1478155885
> Nov 03 14:52:25 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section
> crm_config: OK (rc=0, origin=h122/crmd/109, version=0.497.0)
> Nov 03 14:52:25 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> //node_state[@uname='h122']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> OK (rc=0, origin=h122/crmd/110, version=0.497.0)
> Nov 03 14:52:25 [87774] h123 crmd: ( notify.c:55 ) notice:
> crmd_enable_notifications:Notifications disabled
> Nov 03 14:52:25 [87769] h123 cib: ( cib_file.c:293 ) info:
> cib_file_backup:Archived previous version as
> /var/lib/pacemaker/cib/cib-86.raw
> Nov 03 14:52:25 [87769] h123 cib: ( cib_file.c:423 ) info:
> cib_file_write_with_digest:Wrote version 0.497.0 of the CIB to disk
> (digest: f58e87fbf781a0214e951be264b08a27)
> Nov 03 14:52:25 [87769] h123 cib: ( cib_file.c:442 ) info:
> cib_file_write_with_digest:Reading cluster configuration file
> /var/lib/pacemaker/cib/cib.71JZzY (digest:
> /var/lib/pacemaker/cib/cib.hPhB1b)
> Nov 03 14:52:30 [87769] h123 cib: ( messages.c:239 ) info:
> cib_process_ping:Reporting our current digest to h122:
> e88ff9368e83c51b686a12bd888fc6a5 for 0.497.0 (0x2263e70 0)
> Nov 03 14:52:30 [87771] h123 lrmd: ( lrmd.c:1438 ) info:
> process_lrmd_get_rsc_info:Resource 'rgavhcnP-iOgMD4-Bdht' not found (1
> active resources)
> Nov 03 14:52:30 [87771] h123 lrmd: ( lrmd.c:1413 ) info:
> process_lrmd_rsc_register:Added 'rgavhcnP-iOgMD4-Bdht' to the rsc list
> (2 active resources)
> Nov 03 14:52:30 [87774] h123 crmd: ( lrm.c:2009 ) info:
> do_lrm_rsc_op:Performing key=8:14:7:3c5a583b-d35c-4623-810f-2c63304cb99c
> op=rgavhcnP-iOgMD4-Bdht_monitor_0
> Nov 03 14:52:30 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.497.0 2
> Nov 03 14:52:30 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.497.1 (null)
> Nov 03 14:52:30 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=1
> Nov 03 14:52:30 [87769] h123 cib: ( xml.c:1684 ) info:
> cib_perform_op:++
> /cib/status/node_state[@id='1084759930']/lrm[@id='1084759930']/lrm_resources:
> <lrm_resource id="rgavhcnP-iOgMD4-Bdht" type="fronvm" class="ocf"
> provider="heartbeat"/>
> Nov 03 14:52:30 [87769] h123 cib: ( xml.c:1691 ) info:
> cib_perform_op:++
> <lrm_rsc_op id="rgavhcnP-iOgMD4-Bdht_last_0"
> operation_key="rgavhcnP-iOgMD4-Bdht_monitor_0" operation="monitor"
> crm-debug-origin="do_update_resource" crm_feature_set="3.0.10"
> transition-key="6:14:7:3c5a583b-d35c-4623-810f-2c63304cb99c"
> transition-magic="0:7;6:14:7:3c5a583b-d35c-4623-810f-2c63304cb99c"
> on_node="h122" call-id="50" rc-code=
> Nov 03 14:52:30 [87769] h123 cib: ( xml.c:1691 ) info:
> cib_perform_op:++
> </lrm_resource>
> Nov 03 14:52:30 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section status:
> OK (rc=0, origin=h122/crmd/113, version=0.497.1)
> Nov 03 14:52:30 [87774] h123 crmd: ( lrm.c:2392 ) notice:
> process_lrm_event:Operation rgavhcnP-iOgMD4-Bdht_monitor_0: ok
> (node=h123, call=40, rc=0, cib-update=47, confirmed=true)
> Nov 03 14:52:30 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_modify operation for section status
> to master (origin=local/crmd/47)
> Nov 03 14:52:31 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.497.1 2
> Nov 03 14:52:31 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.497.2 (null)
> Nov 03 14:52:31 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=2
> Nov 03 14:52:31 [87769] h123 cib: ( xml.c:1684 ) info:
> cib_perform_op:++
> /cib/status/node_state[@id='1084759931']/lrm[@id='1084759931']/lrm_resources:
> <lrm_resource id="rgavhcnP-iOgMD4-Bdht" type="fronvm" class="ocf"
> provider="heartbeat"/>
> Nov 03 14:52:31 [87769] h123 cib: ( xml.c:1691 ) info:
> cib_perform_op:++
> <lrm_rsc_op
> id="rgavhcnP-iOgMD4-Bdht_last_failure_0"
> operation_key="rgavhcnP-iOgMD4-Bdht_monitor_0" operation="monitor"
> crm-debug-origin="do_update_resource" crm_feature_set="3.0.10"
> transition-key="8:14:7:3c5a583b-d35c-4623-810f-2c63304cb99c"
> transition-magic="0:0;8:14:7:3c5a583b-d35c-4623-810f-2c63304cb99c"
> on_node="h123" call-id="40"
> Nov 03 14:52:31 [87769] h123 cib: ( xml.c:1691 ) info:
> cib_perform_op:++
> <lrm_rsc_op id="rgavhcnP-iOgMD4-Bdht_last_0"
> operation_key="rgavhcnP-iOgMD4-Bdht_monitor_0" operation="monitor"
> crm-debug-origin="do_update_resource" crm_feature_set="3.0.10"
> transition-key="8:14:7:3c5a583b-d35c-4623-810f-2c63304cb99c"
> transition-magic="0:0;8:14:7:3c5a583b-d35c-4623-810f-2c63304cb99c"
> on_node="h123" call-id="40" rc-code=
> Nov 03 14:52:31 [87769] h123 cib: ( xml.c:1691 ) info:
> cib_perform_op:++
> </lrm_resource>
> Nov 03 14:52:31 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section status:
> OK (rc=0, origin=h123/crmd/47, version=0.497.2)
> Nov 03 14:52:32 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.497.2 2
> Nov 03 14:52:32 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.498.0 (null)
> Nov 03 14:52:32 [87769] h123 cib: ( xml.c:1735 ) info:
> cib_perform_op:--
> /cib/configuration/constraints/rsc_location[@id='cli-prefer-rgavhcnP-iOgMD4-Bdht']
> Nov 03 14:52:32 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @epoch=498, @num_updates=0
> Nov 03 14:52:32 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> constraints: OK (rc=0, origin=h122/crm_resource/3, version=0.498.0)
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( main.c:823 ) info:
> update_cib_stonith_devices_v2:Updating device list from the cib: delete
> rsc_location[@id='cli-prefer-rgavhcnP-iOgMD4-Bdht']
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( main.c:762 ) info:
> cib_devices_update:Updating devices to version 0.498.0
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:162 ) debug:
> unpack_config:STONITH timeout: 120000
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:166 ) debug:
> unpack_config:STONITH of failed nodes is enabled
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:169 ) trace:
> unpack_config:STONITH will reboot nodes
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:173 ) debug:
> unpack_config:Stop all active resources: false
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:182 ) debug:
> unpack_config:Default stickiness: 0
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:219 ) debug:
> unpack_config:On loss of CCM Quorum: Stop ALL resources
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:231 ) trace:
> unpack_config:Orphan resources are stopped
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:235 ) trace:
> unpack_config:Orphan resource actions are stopped
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:239 ) trace:
> unpack_config:Stopped resources are removed from the status section: false
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:243 ) trace:
> unpack_config:Maintenance mode: false
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:251 ) trace:
> unpack_config:By default resources are managed
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:256 ) trace:
> unpack_config:Start failures are handled by failcount
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:265 ) debug:
> unpack_config:Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:268 ) trace:
> unpack_config:Placement strategy: default
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:548 ) trace:
> unpack_nodes:Processing node h122/1084759930
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:313 ) trace:
> create_node:Creating node for entry h122/1084759930
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:572 ) trace:
> unpack_nodes:Done with node h122
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:548 ) trace:
> unpack_nodes:Processing node h123/1084759931
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:313 ) trace:
> create_node:Creating node for entry h123/1084759931
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:572 ) trace:
> unpack_nodes:Done with node h123
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:753 ) trace:
> unpack_resources:Beginning unpack... <clone id=fence-clone... >
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:117 ) trace:
> native_unpack:Processing resource fence:0...
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:555 ) trace:
> native_print:Added fence(stonith:fence_agent):Stopped
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:117 ) trace:
> native_unpack:Processing resource fence:1...
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:555 ) trace:
> native_print:Added fence(stonith:fence_agent):Stopped
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( unpack.c:753 ) trace:
> unpack_resources:Beginning unpack... <primitive id=rgavhcnP-iOgMD4-Bdht... >
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:117 ) trace:
> native_unpack:Processing resource rgavhcnP-iOgMD4-Bdht...
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:555 ) trace:
> native_print:Added rgavhcnP-iOgMD4-Bdht(ocf::heartbeat:fronvm):Stopped
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( allocate.c:1027 ) trace:
> stage2:Applying placement constraints
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( allocate.c:585 ) trace:
> apply_placement_constraints:Applying constraints...
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying
> loc-tEJVJGWl-ifcTSp-Ta1v-rgavhcnP-iOgMD4-Bdht (Unknown) to
> rgavhcnP-iOgMD4-Bdht
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:rgavhcnP-iOgMD4-Bdht + h123 : 2
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying
> loc-paPupXaC-Ns5mnb-Xdfe-rgavhcnP-iOgMD4-Bdht (Unknown) to
> rgavhcnP-iOgMD4-Bdht
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:rgavhcnP-iOgMD4-Bdht + h122 : 1
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:rgavhcnP-iOgMD4-Bdht + h123 : 2
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h123 (Unknown) to fence-clone
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence-clone + h123 : 100
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h123 (Unknown) to fence:0
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:0 + h123 : 100
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h123 (Unknown) to fence:1
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:1 + h123 : 100
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h122 (Unknown) to fence-clone
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence-clone + h122 : 100
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence-clone + h123 : 100
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h122 (Unknown) to fence:0
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:0 + h122 : 100
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:0 + h123 : 100
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h122 (Unknown) to fence:1
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:1 + h122 : 100
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:1 + h123 : 100
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( allocate.c:2684 ) trace:
> cleanup_alloc_calculations:deleting 0 order cons: (nil)
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( allocate.c:2689 ) trace:
> cleanup_alloc_calculations:deleting 4 node cons: 0x23ed4a0
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( allocate.c:2694 ) trace:
> cleanup_alloc_calculations:deleting 0 inter-resource cons: (nil)
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( allocate.c:2699 ) trace:
> cleanup_alloc_calculations:deleting 0 ticket deps: (nil)
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:662 ) trace:
> native_free:Freeing resource action list (not the data)
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:662 ) trace:
> native_free:Freeing resource action list (not the data)
> Nov 03 14:52:32 [87770] h123 stonith-ng: ( native.c:662 ) trace:
> native_free:Freeing resource action list (not the data)
> Nov 03 14:52:32 [87769] h123 cib: ( cib_file.c:293 ) info:
> cib_file_backup:Archived previous version as
> /var/lib/pacemaker/cib/cib-87.raw
> Nov 03 14:52:32 [87769] h123 cib: ( cib_file.c:423 ) info:
> cib_file_write_with_digest:Wrote version 0.498.0 of the CIB to disk
> (digest: 402c2c1fe5d4130563acaf5225168dc8)
> Nov 03 14:52:32 [87769] h123 cib: ( cib_file.c:442 ) info:
> cib_file_write_with_digest:Reading cluster configuration file
> /var/lib/pacemaker/cib/cib.NMs2tP (digest:
> /var/lib/pacemaker/cib/cib.Iq752u)
> Nov 03 14:52:38 [87769] h123 cib: ( messages.c:239 ) info:
> cib_process_ping:Reporting our current digest to h122:
> 9c276159d8b4e6e1d1bbbdce096370d6 for 0.498.0 (0x1eec1d0 0)
> Nov 03 14:52:38 [87774] h123 crmd: ( lrm.c:2009 ) info:
> do_lrm_rsc_op:Performing
> key=17:15:0:3c5a583b-d35c-4623-810f-2c63304cb99c
> op=rgavhcnP-iOgMD4-Bdht_monitor_120000
> Nov 03 14:52:38 [87774] h123 crmd: ( lrm.c:2392 ) info:
> process_lrm_event:Operation rgavhcnP-iOgMD4-Bdht_monitor_120000: ok
> (node=h123, call=41, rc=0, cib-update=48, confirmed=false)
> Nov 03 14:52:38 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_modify operation for section status
> to master (origin=local/crmd/48)
> Nov 03 14:52:38 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.498.0 2
> Nov 03 14:52:38 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.498.1 (null)
> Nov 03 14:52:38 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=1
> Nov 03 14:52:38 [87769] h123 cib: ( xml.c:1684 ) info:
> cib_perform_op:++
> /cib/status/node_state[@id='1084759931']/lrm[@id='1084759931']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> <lrm_rsc_op id="rgavhcnP-iOgMD4-Bdht_monitor_120000"
> operation_key="rgavhcnP-iOgMD4-Bdht_monitor_120000" operation="monitor"
> crm-debug-origin="do_update_resource" crm_feature_set="3.0.10"
> transition-key="17:15:0:3c5a583b-d35c-4623-810f-2c63304cb99c"
> transition-magic="0:0;17:15:0:3c5a583b-d35c-4623-8
> Nov 03 14:52:38 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section status:
> OK (rc=0, origin=h123/crmd/48, version=0.498.1)
> Nov 03 14:52:43 [87769] h123 cib: ( messages.c:239 ) info:
> cib_process_ping:Reporting our current digest to h122:
> 9c64f64c91f86408e01b4048c2ec1151 for 0.498.1 (0x1eec1d0 0)
> Nov 03 14:52:52 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_modify operation for section
> resources to master (origin=local/crm_resource/6)
> Nov 03 14:52:52 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.498.1 2
> Nov 03 14:52:52 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.499.0 (null)
> Nov 03 14:52:52 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @epoch=499, @num_updates=0
> Nov 03 14:52:52 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+
> /cib/configuration/resources/primitive[@id='rgavhcnP-iOgMD4-Bdht']/instance_attributes[@id='rgavhcnP-iOgMD4-Bdht-instance_attributes']/nvpair[@id='rgavhcnP-iOgMD4-Bdht-instance_attributes-config']:
> @value=/mnt/eec16e46a2a2a5c93be445c09c41c221a
> Nov 03 14:52:52 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section
> resources: OK (rc=0, origin=h123/crm_resource/6, version=0.499.0)
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( main.c:823 ) info:
> update_cib_stonith_devices_v2:Updating device list from the cib: modify
> nvpair[@id='rgavhcnP-iOgMD4-Bdht-instance_attributes-config']
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( main.c:762 ) info:
> cib_devices_update:Updating devices to version 0.499.0
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:162 ) debug:
> unpack_config:STONITH timeout: 120000
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:166 ) debug:
> unpack_config:STONITH of failed nodes is enabled
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:169 ) trace:
> unpack_config:STONITH will reboot nodes
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:173 ) debug:
> unpack_config:Stop all active resources: false
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:182 ) debug:
> unpack_config:Default stickiness: 0
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:219 ) debug:
> unpack_config:On loss of CCM Quorum: Stop ALL resources
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:231 ) trace:
> unpack_config:Orphan resources are stopped
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:235 ) trace:
> unpack_config:Orphan resource actions are stopped
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:239 ) trace:
> unpack_config:Stopped resources are removed from the status section: false
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:243 ) trace:
> unpack_config:Maintenance mode: false
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:251 ) trace:
> unpack_config:By default resources are managed
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:256 ) trace:
> unpack_config:Start failures are handled by failcount
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:265 ) debug:
> unpack_config:Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:268 ) trace:
> unpack_config:Placement strategy: default
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:548 ) trace:
> unpack_nodes:Processing node h122/1084759930
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:313 ) trace:
> create_node:Creating node for entry h122/1084759930
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:572 ) trace:
> unpack_nodes:Done with node h122
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:548 ) trace:
> unpack_nodes:Processing node h123/1084759931
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:313 ) trace:
> create_node:Creating node for entry h123/1084759931
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:572 ) trace:
> unpack_nodes:Done with node h123
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:753 ) trace:
> unpack_resources:Beginning unpack... <clone id=fence-clone... >
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:117 ) trace:
> native_unpack:Processing resource fence:0...
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:555 ) trace:
> native_print:Added fence(stonith:fence_agent):Stopped
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:117 ) trace:
> native_unpack:Processing resource fence:1...
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:555 ) trace:
> native_print:Added fence(stonith:fence_agent):Stopped
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( unpack.c:753 ) trace:
> unpack_resources:Beginning unpack... <primitive id=rgavhcnP-iOgMD4-Bdht... >
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:117 ) trace:
> native_unpack:Processing resource rgavhcnP-iOgMD4-Bdht...
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:555 ) trace:
> native_print:Added rgavhcnP-iOgMD4-Bdht(ocf::heartbeat:fronvm):Stopped
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( allocate.c:1027 ) trace:
> stage2:Applying placement constraints
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( allocate.c:585 ) trace:
> apply_placement_constraints:Applying constraints...
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying
> loc-tEJVJGWl-ifcTSp-Ta1v-rgavhcnP-iOgMD4-Bdht (Unknown) to
> rgavhcnP-iOgMD4-Bdht
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:rgavhcnP-iOgMD4-Bdht + h123 : 2
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying
> loc-paPupXaC-Ns5mnb-Xdfe-rgavhcnP-iOgMD4-Bdht (Unknown) to
> rgavhcnP-iOgMD4-Bdht
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:rgavhcnP-iOgMD4-Bdht + h122 : 1
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:rgavhcnP-iOgMD4-Bdht + h123 : 2
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h123 (Unknown) to fence-clone
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence-clone + h123 : 100
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h123 (Unknown) to fence:0
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:0 + h123 : 100
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h123 (Unknown) to fence:1
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:1 + h123 : 100
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h122 (Unknown) to fence-clone
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence-clone + h122 : 100
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence-clone + h123 : 100
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h122 (Unknown) to fence:0
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:0 + h122 : 100
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:0 + h123 : 100
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2021 ) trace:
> native_rsc_location:Applying loc-fence-h122 (Unknown) to fence:1
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:1 + h122 : 100
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:2068 ) trace:
> native_rsc_location:fence:1 + h123 : 100
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( allocate.c:2684 ) trace:
> cleanup_alloc_calculations:deleting 0 order cons: (nil)
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( allocate.c:2689 ) trace:
> cleanup_alloc_calculations:deleting 4 node cons: 0x23ed900
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( allocate.c:2694 ) trace:
> cleanup_alloc_calculations:deleting 0 inter-resource cons: (nil)
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( allocate.c:2699 ) trace:
> cleanup_alloc_calculations:deleting 0 ticket deps: (nil)
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:662 ) trace:
> native_free:Freeing resource action list (not the data)
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:662 ) trace:
> native_free:Freeing resource action list (not the data)
> Nov 03 14:52:52 [87770] h123 stonith-ng: ( native.c:662 ) trace:
> native_free:Freeing resource action list (not the data)
> Nov 03 14:52:52 [87769] h123 cib: ( cib_file.c:293 ) info:
> cib_file_backup:Archived previous version as
> /var/lib/pacemaker/cib/cib-88.raw
> Nov 03 14:52:52 [87769] h123 cib: ( cib_file.c:423 ) info:
> cib_file_write_with_digest:Wrote version 0.499.0 of the CIB to disk
> (digest: f9f54a00c0e381c5a8c573d7f292ec39)
> Nov 03 14:52:52 [87769] h123 cib: ( cib_file.c:442 ) info:
> cib_file_write_with_digest:Reading cluster configuration file
> /var/lib/pacemaker/cib/cib.qgjoNM (digest:
> /var/lib/pacemaker/cib/cib.W8UiDF)
> Nov 03 14:52:53 [87774] h123 crmd: ( throttle.c:430 ) info:
> throttle_handle_load:Moderate CPU load detected: 0.990000
> Nov 03 14:52:53 [87774] h123 crmd: ( throttle.c:531 ) info:
> throttle_send_command:New throttle mode: 0010 (was 0000)
> Nov 03 14:52:57 [87769] h123 cib: ( messages.c:239 ) info:
> cib_process_ping:Reporting our current digest to h122:
> f83fef3d0ea7c60b279dc664ef8bf13e for 0.499.0 (0x2263340 0)
> Nov 03 14:52:57 [87771] h123 lrmd: ( services.c:463 ) info:
> cancel_recurring_action:Cancelling ocf operation
> rgavhcnP-iOgMD4-Bdht_monitor_120000
> Nov 03 14:52:57 [87774] h123 crmd: ( lrm.c:2009 ) info:
> do_lrm_rsc_op:Performing
> key=18:16:0:3c5a583b-d35c-4623-810f-2c63304cb99c
> op=rgavhcnP-iOgMD4-Bdht_stop_0
> Nov 03 14:52:57 [87771] h123 lrmd: ( lrmd.c:131 ) info:
> log_execute:executing - rsc:rgavhcnP-iOgMD4-Bdht action:stop call_id:43
> Nov 03 14:52:57 [87774] h123 crmd: ( lrm.c:2385 ) info:
> process_lrm_event:Operation rgavhcnP-iOgMD4-Bdht_monitor_120000:
> Cancelled (node=h123, call=41, confirmed=true)
> Nov 03 14:52:57 [87774] h123 crmd: ( messages.c:639 ) info:
> handle_failcount_op:Removing failcount for rgavhcnP-iOgMD4-Bdht
> Nov 03 14:52:57 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_delete operation for section
> //node_state[@uname='h123']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']/lrm_rsc_op[@id='rgavhcnP-iOgMD4-Bdht_last_failure_0']
> to master (origin=local/crmd/49)
> Nov 03 14:52:57 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.499.0 2
> Nov 03 14:52:57 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.499.1 (null)
> Nov 03 14:52:57 [87769] h123 cib: ( xml.c:1735 ) info:
> cib_perform_op:--
> /cib/status/node_state[@id='1084759931']/lrm[@id='1084759931']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']/lrm_rsc_op[@id='rgavhcnP-iOgMD4-Bdht_last_failure_0']
> Nov 03 14:52:57 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=1
> Nov 03 14:52:57 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> //node_state[@uname='h123']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']/lrm_rsc_op[@id='rgavhcnP-iOgMD4-Bdht_last_failure_0']:
> OK (rc=0, origin=h123/crmd/49, version=0.499.1)
> Nov 03 14:52:58 [87771] h123 lrmd: ( lrmd.c:113 ) info:
> log_finished:finished - rsc:rgavhcnP-iOgMD4-Bdht action:stop call_id:43
> pid:119448 exit-code:0 exec-time:1109ms queue-time:0ms
> Nov 03 14:52:58 [87774] h123 crmd: ( lrm.c:2392 ) notice:
> process_lrm_event:Operation rgavhcnP-iOgMD4-Bdht_stop_0: ok (node=h123,
> call=43, rc=0, cib-update=50, confirmed=true)
> Nov 03 14:52:58 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_modify operation for section status
> to master (origin=local/crmd/50)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.499.1 2
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.499.2 (null)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=2
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+
> /cib/status/node_state[@id='1084759931']/lrm[@id='1084759931']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']/lrm_rsc_op[@id='rgavhcnP-iOgMD4-Bdht_last_0']:
> @operation_key=rgavhcnP-iOgMD4-Bdht_stop_0, @operation=stop,
> @transition-key=18:16:0:3c5a583b-d35c-4623-810f-2c63304cb99c,
> @transition-magic=0:0;18:16:0:3c5a583b-d35c-4623-810f-2c63304cb99c,
> @call-id=43, @last-run=1478155977, @last-rc-change=1478155977,
> Nov 03 14:52:58 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section status:
> OK (rc=0, origin=h123/crmd/50, version=0.499.2)
> Nov 03 14:52:58 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_delete operation for section
> //node_state[@uname='h123']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht'] to
> master (origin=local/crmd/51)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.499.2 2
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.499.3 (null)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1735 ) info:
> cib_perform_op:--
> /cib/status/node_state[@id='1084759931']/lrm[@id='1084759931']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=3
> Nov 03 14:52:58 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> //node_state[@uname='h123']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> OK (rc=0, origin=h123/crmd/51, version=0.499.2)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.499.2 2
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.499.3 (null)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1735 ) info:
> cib_perform_op:--
> /cib/status/node_state[@id='1084759930']/lrm[@id='1084759930']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=3
> Nov 03 14:52:58 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> //node_state[@uname='h122']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> OK (rc=0, origin=h122/crmd/116, version=0.499.2)
> Nov 03 14:52:58 [87770] h123 stonith-ng: ( xml.c:2055 ) info:
> xml_patch_version_check:Current num_updates is too high (3 > 2)
> Nov 03 14:52:58 [87774] h123 crmd: ( lrm.c:1272 ) info:
> delete_resource:Removing resource rgavhcnP-iOgMD4-Bdht for
> f169d333-7819-4982-9640-5c914763da9e (root) on h123
> Nov 03 14:52:58 [87774] h123 crmd: ( lrm.c:890 ) info:
> notify_deleted:Notifying f169d333-7819-4982-9640-5c914763da9e on h123
> that rgavhcnP-iOgMD4-Bdht was deleted
> Nov 03 14:52:58 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_delete operation for section
> //node_state[@uname='h123']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht'] to
> master (origin=local/crmd/52)
> Nov 03 14:52:58 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_modify operation for section
> crm_config to master (origin=local/crmd/54)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.499.2 2
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.499.3 (null)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1735 ) info:
> cib_perform_op:--
> /cib/status/node_state[@id='1084759931']/lrm[@id='1084759931']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=3
> Nov 03 14:52:58 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> //node_state[@uname='h123']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> OK (rc=0, origin=h123/crmd/52, version=0.499.3)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.499.3 2
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.500.0 (null)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @epoch=500, @num_updates=0
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+
> /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-last-lrm-refresh']:
> @value=1478155978
> Nov 03 14:52:58 [87770] h123 stonith-ng: ( xml.c:2055 ) info:
> xml_patch_version_check:Current num_updates is too high (3 > 2)
> Nov 03 14:52:58 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section
> crm_config: OK (rc=0, origin=h123/crmd/54, version=0.500.0)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.500.0 2
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.500.1 (null)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1735 ) info:
> cib_perform_op:--
> /cib/status/node_state[@id='1084759930']/lrm[@id='1084759930']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=1
> Nov 03 14:52:58 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_delete operation for section
> //node_state[@uname='h122']//lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> OK (rc=0, origin=h122/crmd/117, version=0.500.1)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.500.1 2
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.501.0 (null)
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @epoch=501, @num_updates=0
> Nov 03 14:52:58 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+
> /cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-last-lrm-refresh']:
> @value=1478155918
> Nov 03 14:52:58 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section
> crm_config: OK (rc=0, origin=h122/crmd/119, version=0.501.0)
> Nov 03 14:52:58 [87774] h123 crmd: ( notify.c:55 ) notice:
> crmd_enable_notifications:Notifications disabled
> Nov 03 14:52:58 [87774] h123 crmd: ( notify.c:55 ) notice:
> crmd_enable_notifications:Notifications disabled
> Nov 03 14:52:58 [87769] h123 cib: ( cib_file.c:293 ) info:
> cib_file_backup:Archived previous version as
> /var/lib/pacemaker/cib/cib-89.raw
> Nov 03 14:52:58 [87769] h123 cib: ( cib_file.c:423 ) info:
> cib_file_write_with_digest:Wrote version 0.501.0 of the CIB to disk
> (digest: 58f4aeff25c5a0c1f0dbbe42dc889003)
> Nov 03 14:52:58 [87769] h123 cib: ( cib_file.c:442 ) info:
> cib_file_write_with_digest:Reading cluster configuration file
> /var/lib/pacemaker/cib/cib.qGD8Tv (digest:
> /var/lib/pacemaker/cib/cib.IKpaJM)
> Nov 03 14:53:03 [87769] h123 cib: ( messages.c:239 ) info:
> cib_process_ping:Reporting our current digest to h122:
> ecb9a13c9a6bdec31e17e66366e5dbfb for 0.501.0 (0x1cf9480 0)
> Nov 03 14:53:03 [87771] h123 lrmd: ( lrmd.c:1438 ) info:
> process_lrmd_get_rsc_info:Resource 'rgavhcnP-iOgMD4-Bdht' not found (1
> active resources)
> Nov 03 14:53:03 [87771] h123 lrmd: ( lrmd.c:1413 ) info:
> process_lrmd_rsc_register:Added 'rgavhcnP-iOgMD4-Bdht' to the rsc list
> (2 active resources)
> Nov 03 14:53:03 [87774] h123 crmd: ( lrm.c:2009 ) info:
> do_lrm_rsc_op:Performing key=8:17:7:3c5a583b-d35c-4623-810f-2c63304cb99c
> op=rgavhcnP-iOgMD4-Bdht_monitor_0
> Nov 03 14:53:04 [87774] h123 crmd: ( lrm.c:2392 ) notice:
> process_lrm_event:Operation rgavhcnP-iOgMD4-Bdht_monitor_0: not running
> (node=h123, call=48, rc=7, cib-update=57, confirmed=true)
> Nov 03 14:53:04 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_modify operation for section status
> to master (origin=local/crmd/57)
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.501.0 2
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.501.1 (null)
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=1
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1684 ) info:
> cib_perform_op:++
> /cib/status/node_state[@id='1084759931']/lrm[@id='1084759931']/lrm_resources:
> <lrm_resource id="rgavhcnP-iOgMD4-Bdht" type="fronvm" class="ocf"
> provider="heartbeat"/>
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1691 ) info:
> cib_perform_op:++
> <lrm_rsc_op id="rgavhcnP-iOgMD4-Bdht_last_0"
> operation_key="rgavhcnP-iOgMD4-Bdht_monitor_0" operation="monitor"
> crm-debug-origin="do_update_resource" crm_feature_set="3.0.10"
> transition-key="8:17:7:3c5a583b-d35c-4623-810f-2c63304cb99c"
> transition-magic="0:7;8:17:7:3c5a583b-d35c-4623-810f-2c63304cb99c"
> on_node="h123" call-id="48" rc-code=
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1691 ) info:
> cib_perform_op:++
> </lrm_resource>
> Nov 03 14:53:04 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section status:
> OK (rc=0, origin=h123/crmd/57, version=0.501.1)
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.501.1 2
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.501.2 (null)
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=2
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1684 ) info:
> cib_perform_op:++
> /cib/status/node_state[@id='1084759930']/lrm[@id='1084759930']/lrm_resources:
> <lrm_resource id="rgavhcnP-iOgMD4-Bdht" type="fronvm" class="ocf"
> provider="heartbeat"/>
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1691 ) info:
> cib_perform_op:++
> <lrm_rsc_op id="rgavhcnP-iOgMD4-Bdht_last_0"
> operation_key="rgavhcnP-iOgMD4-Bdht_monitor_0" operation="monitor"
> crm-debug-origin="do_update_resource" crm_feature_set="3.0.10"
> transition-key="6:17:7:3c5a583b-d35c-4623-810f-2c63304cb99c"
> transition-magic="0:7;6:17:7:3c5a583b-d35c-4623-810f-2c63304cb99c"
> on_node="h122" call-id="55" rc-code=
> Nov 03 14:53:04 [87769] h123 cib: ( xml.c:1691 ) info:
> cib_perform_op:++
> </lrm_resource>
> Nov 03 14:53:04 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section status:
> OK (rc=0, origin=h122/crmd/123, version=0.501.2)
> Nov 03 14:53:04 [87774] h123 crmd: ( lrm.c:2009 ) info:
> do_lrm_rsc_op:Performing
> key=17:17:0:3c5a583b-d35c-4623-810f-2c63304cb99c
> op=rgavhcnP-iOgMD4-Bdht_start_0
> Nov 03 14:53:04 [87771] h123 lrmd: ( lrmd.c:131 ) info:
> log_execute:executing - rsc:rgavhcnP-iOgMD4-Bdht action:start call_id:49
> Nov 03 14:53:09 [87769] h123 cib: ( messages.c:239 ) info:
> cib_process_ping:Reporting our current digest to h122:
> 33922f7663811402c33c00d676b92ef2 for 0.501.2 (0x1cf9480 0)
> Nov 03 14:53:23 [87774] h123 crmd: ( throttle.c:430 ) info:
> throttle_handle_load:Moderate CPU load detected: 0.880000
> Nov 03 14:53:25 [87771] h123 lrmd: ( lrmd.c:113 ) info:
> log_finished:finished - rsc:rgavhcnP-iOgMD4-Bdht action:start call_id:49
> pid:119661 exit-code:0 exec-time:21311ms queue-time:0ms
> Nov 03 14:53:25 [87774] h123 crmd: (services_lin:567 ) info:
> action_synced_wait:Managed fronvm_meta-data_0 process 120142 exited with
> rc=0
> Nov 03 14:53:25 [87774] h123 crmd: ( lrm.c:2392 ) notice:
> process_lrm_event:Operation rgavhcnP-iOgMD4-Bdht_start_0: ok (node=h123,
> call=49, rc=0, cib-update=58, confirmed=true)
> Nov 03 14:53:25 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_modify operation for section status
> to master (origin=local/crmd/58)
> Nov 03 14:53:25 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.501.2 2
> Nov 03 14:53:25 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.501.3 (null)
> Nov 03 14:53:25 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=3
> Nov 03 14:53:25 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+
> /cib/status/node_state[@id='1084759931']/lrm[@id='1084759931']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']/lrm_rsc_op[@id='rgavhcnP-iOgMD4-Bdht_last_0']:
> @operation_key=rgavhcnP-iOgMD4-Bdht_start_0, @operation=start,
> @transition-key=17:17:0:3c5a583b-d35c-4623-810f-2c63304cb99c,
> @transition-magic=0:0;17:17:0:3c5a583b-d35c-4623-810f-2c63304cb99c,
> @call-id=49, @rc-code=0, @last-run=1478155984, @last-rc-chang
> Nov 03 14:53:25 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section status:
> OK (rc=0, origin=h123/crmd/58, version=0.501.3)
> Nov 03 14:53:25 [87774] h123 crmd: ( lrm.c:2009 ) info:
> do_lrm_rsc_op:Performing
> key=18:17:0:3c5a583b-d35c-4623-810f-2c63304cb99c
> op=rgavhcnP-iOgMD4-Bdht_monitor_120000
> Nov 03 14:53:26 [87774] h123 crmd: ( lrm.c:2392 ) info:
> process_lrm_event:Operation rgavhcnP-iOgMD4-Bdht_monitor_120000: ok
> (node=h123, call=50, rc=0, cib-update=59, confirmed=false)
> Nov 03 14:53:26 [87769] h123 cib: ( callbacks.c:1015 ) info:
> cib_process_request:Forwarding cib_modify operation for section status
> to master (origin=local/crmd/59)
> Nov 03 14:53:26 [87769] h123 cib: ( xml.c:1660 ) info:
> cib_perform_op:Diff: --- 0.501.3 2
> Nov 03 14:53:26 [87769] h123 cib: ( xml.c:1662 ) info:
> cib_perform_op:Diff: +++ 0.501.4 (null)
> Nov 03 14:53:26 [87769] h123 cib: ( xml.c:1728 ) info:
> cib_perform_op:+ /cib: @num_updates=4
> Nov 03 14:53:26 [87769] h123 cib: ( xml.c:1684 ) info:
> cib_perform_op:++
> /cib/status/node_state[@id='1084759931']/lrm[@id='1084759931']/lrm_resources/lrm_resource[@id='rgavhcnP-iOgMD4-Bdht']:
> <lrm_rsc_op id="rgavhcnP-iOgMD4-Bdht_monitor_120000"
> operation_key="rgavhcnP-iOgMD4-Bdht_monitor_120000" operation="monitor"
> crm-debug-origin="do_update_resource" crm_feature_set="3.0.10"
> transition-key="18:17:0:3c5a583b-d35c-4623-810f-2c63304cb99c"
> transition-magic="0:0;18:17:0:3c5a583b-d35c-4623-8
> Nov 03 14:53:26 [87769] h123 cib: ( callbacks.c:1079 ) info:
> cib_process_request:Completed cib_modify operation for section status:
> OK (rc=0, origin=h123/crmd/59, version=0.501.4)
> Nov 03 14:53:31 [87769] h123 cib: ( messages.c:239 ) info:
> cib_process_ping:Reporting our current digest to h122:
> c65b5df85fb72339a2775371cb627cec for 0.501.4 (0x1cf9480 0)
>
>
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
More information about the Users
mailing list