Oct 15 15:15:47 vm1 corosync[14853]:   [MAIN  ] main.c:main:1171 Corosync Cluster Engine ('2.3.2.4-805b3'): started and ready to provide service.
Oct 15 15:15:47 vm1 corosync[14853]:   [MAIN  ] main.c:main:1172 Corosync built-in features: watchdog upstart snmp pie relro bindnow
Oct 15 15:15:47 vm1 corosync[14855]:   [TOTEM ] totemnet.c:totemnet_instance_initialize:242 Initializing transport (UDP/IP Multicast).
Oct 15 15:15:47 vm1 corosync[14855]:   [TOTEM ] totemcrypto.c:init_nss:579 Initializing transmit/receive security (NSS) crypto: aes256 hash: sha1
Oct 15 15:15:47 vm1 corosync[14855]:   [TOTEM ] totemnet.c:totemnet_instance_initialize:242 Initializing transport (UDP/IP Multicast).
Oct 15 15:15:47 vm1 corosync[14855]:   [TOTEM ] totemcrypto.c:init_nss:579 Initializing transmit/receive security (NSS) crypto: aes256 hash: sha1
Oct 15 15:15:47 vm1 corosync[14855]:   [TOTEM ] totemudp.c:timer_function_netif_check_timeout:670 The network interface [192.168.101.141] is now up.
Oct 15 15:15:47 vm1 corosync[14855]:   [SERV  ] service.c:corosync_service_link_and_init:174 Service engine loaded: corosync configuration map access [0]
Oct 15 15:15:47 vm1 corosync[14855]:   [QB    ] ipc_setup.c:qb_ipcs_us_publish:374 server name: cmap
Oct 15 15:15:47 vm1 corosync[14855]:   [SERV  ] service.c:corosync_service_link_and_init:174 Service engine loaded: corosync configuration service [1]
Oct 15 15:15:47 vm1 corosync[14855]:   [QB    ] ipc_setup.c:qb_ipcs_us_publish:374 server name: cfg
Oct 15 15:15:47 vm1 corosync[14855]:   [SERV  ] service.c:corosync_service_link_and_init:174 Service engine loaded: corosync cluster closed process group service v1.01 [2]
Oct 15 15:15:47 vm1 corosync[14855]:   [QB    ] ipc_setup.c:qb_ipcs_us_publish:374 server name: cpg
Oct 15 15:15:47 vm1 corosync[14855]:   [SERV  ] service.c:corosync_service_link_and_init:174 Service engine loaded: corosync profile loading service [4]
Oct 15 15:15:47 vm1 corosync[14855]:   [WD    ] wd.c:setup_watchdog:651 Watchdog is now been tickled by corosync.
Oct 15 15:15:47 vm1 corosync[14855]:   [WD    ] wd.c:wd_scan_resources:580 no resources configured.
Oct 15 15:15:47 vm1 corosync[14855]:   [SERV  ] service.c:corosync_service_link_and_init:174 Service engine loaded: corosync watchdog service [7]
Oct 15 15:15:47 vm1 corosync[14855]:   [QUORUM] vsf_quorum.c:quorum_exec_init_fn:274 Using quorum provider corosync_votequorum
Oct 15 15:15:47 vm1 corosync[14855]:   [VOTEQ ] votequorum.c:are_we_quorate:744 Waiting for all cluster members. Current votes: 1 expected_votes: 2
Oct 15 15:15:47 vm1 corosync[14855]:   [SERV  ] service.c:corosync_service_link_and_init:174 Service engine loaded: corosync vote quorum service v1.0 [5]
Oct 15 15:15:47 vm1 corosync[14855]:   [QB    ] ipc_setup.c:qb_ipcs_us_publish:374 server name: votequorum
Oct 15 15:15:47 vm1 corosync[14855]:   [SERV  ] service.c:corosync_service_link_and_init:174 Service engine loaded: corosync cluster quorum service v0.1 [3]
Oct 15 15:15:47 vm1 corosync[14855]:   [QB    ] ipc_setup.c:qb_ipcs_us_publish:374 server name: quorum
Oct 15 15:15:47 vm1 corosync[14855]:   [TOTEM ] totemudp.c:timer_function_netif_check_timeout:670 The network interface [192.168.102.141] is now up.
Oct 15 15:15:48 vm1 corosync[14855]:   [TOTEM ] totemsrp.c:memb_state_operational_enter:1966 A new membership (192.168.101.141:4) was formed. Members joined: -1062705779
Oct 15 15:15:48 vm1 corosync[14855]:   [VOTEQ ] votequorum.c:are_we_quorate:744 Waiting for all cluster members. Current votes: 1 expected_votes: 2
Oct 15 15:15:48 vm1 corosync[14855]:   [VOTEQ ] votequorum.c:are_we_quorate:744 Waiting for all cluster members. Current votes: 1 expected_votes: 2
Oct 15 15:15:48 vm1 corosync[14855]:   [VOTEQ ] votequorum.c:are_we_quorate:744 Waiting for all cluster members. Current votes: 1 expected_votes: 2
Oct 15 15:15:48 vm1 corosync[14855]:   [QUORUM] vsf_quorum.c:log_view_list:132 Members[1]: -1062705779
Oct 15 15:15:48 vm1 corosync[14855]:   [MAIN  ] main.c:corosync_sync_completed:276 Completed service synchronization, ready to provide service.
Oct 15 15:15:49 vm1 corosync[14855]:   [TOTEM ] totemsrp.c:memb_state_operational_enter:1966 A new membership (192.168.101.141:12) was formed. Members joined: -1062705778 -1062705777
Oct 15 15:15:49 vm1 corosync[14855]:   [QUORUM] vsf_quorum.c:quorum_api_set_quorum:148 This node is within the primary component and will provide service.
Oct 15 15:15:49 vm1 corosync[14855]:   [QUORUM] vsf_quorum.c:log_view_list:132 Members[3]: -1062705779 -1062705778 -1062705777
Oct 15 15:15:49 vm1 corosync[14855]:   [MAIN  ] main.c:corosync_sync_completed:276 Completed service synchronization, ready to provide service.
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: crm_add_logfile: Additional logging available in /var/log/ha-debug
Oct 15 15:15:50 vm1 pacemakerd[14865]:    debug: crm_update_callsites: Enabling callsites based on priority=7, files=(null), functions=(null), formats=(null), tags=(null)
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/root
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_ipc_connect: Could not establish pacemakerd connection: Connection refused (111)
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: get_cluster_type: Detected an active 'corosync' cluster
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: mcp_read_config: Reading configure for stack: corosync
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: mcp_read_config: Configured corosync to accept connections from group 492: OK (1)
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: crm_add_logfile: Additional logging available in /var/log/ha-debug
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: main: Starting Pacemaker 1.1.11-0.284.6a5e863.git.el6 (Build: 6a5e863):  generated-manpages agent-manpages ascii-docs ncurses libqb-logging libqb-ipc lha-fencing nagios  corosync-native snmp
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: main: Maximum core file size is: 18446744073709551615
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: qb_ipcs_us_publish: server name: pacemakerd
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_get_peer: Created entry 24ff3df3-060a-4a43-b8d7-5e0a3f9c874e/0x962120 for node (null)/3232261517 (1 total)
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_get_peer: Node 3232261517 has uuid 3232261517
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3232261517] - corosync-cpg is now online
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: cluster_connect_quorum: Quorum acquired
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261517
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: get_node_name: Defaulting to uname -n for the local corosync node name
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_get_peer: Node 3232261517 is now known as vm1
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: start_child: Using uid=496 and group=492 for process cib
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: start_child: Forked child 14869 for process cib
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: start_child: Forked child 14870 for process stonith-ng
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: start_child: Forked child 14871 for process lrmd
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: start_child: Using uid=496 and group=492 for process attrd
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: start_child: Forked child 14872 for process attrd
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: start_child: Using uid=496 and group=492 for process pengine
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: start_child: Forked child 14873 for process pengine
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: start_child: Using uid=496 and group=492 for process crmd
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: start_child: Forked child 14874 for process crmd
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: main: Starting mainloop
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: pcmk_quorum_notification: Membership 12: quorum retained (3)
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: crm_update_peer_state: pcmk_quorum_notification: Node vm1[3232261517] - state is now member (was (null))
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_get_peer: Created entry 59e9d29e-9a67-417b-9412-5ee33908531a/0xa63cc0 for node (null)/3232261518 (2 total)
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_get_peer: Node 3232261518 has uuid 3232261518
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: pcmk_quorum_notification: Obtaining name for new node 3232261518
Oct 15 15:15:50 vm1 cib[14869]:   notice: crm_add_logfile: Additional logging available in /var/log/ha-debug
Oct 15 15:15:50 vm1 cib[14869]:    debug: crm_update_callsites: Enabling callsites based on priority=7, files=(null), functions=(null), formats=(null), tags=(null)
Oct 15 15:15:50 vm1 cib[14869]:     info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster
Oct 15 15:15:50 vm1 stonith-ng[14870]:   notice: crm_add_logfile: Additional logging available in /var/log/ha-debug
Oct 15 15:15:50 vm1 stonith-ng[14870]:    debug: crm_update_callsites: Enabling callsites based on priority=7, files=(null), functions=(null), formats=(null), tags=(null)
Oct 15 15:15:50 vm1 cib[14869]:   notice: main: Using new config location: /var/lib/pacemaker/cib
Oct 15 15:15:50 vm1 cib[14869]:     info: get_cluster_type: Verifying cluster type: 'corosync'
Oct 15 15:15:50 vm1 stonith-ng[14870]:     info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/root
Oct 15 15:15:50 vm1 stonith-ng[14870]:     info: get_cluster_type: Verifying cluster type: 'corosync'
Oct 15 15:15:50 vm1 stonith-ng[14870]:     info: get_cluster_type: Assuming an active 'corosync' cluster
Oct 15 15:15:50 vm1 stonith-ng[14870]:   notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync
Oct 15 15:15:50 vm1 cib[14869]:     info: get_cluster_type: Assuming an active 'corosync' cluster
Oct 15 15:15:50 vm1 cib[14869]:     info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.xml (digest: /var/lib/pacemaker/cib/cib.xml.sig)
Oct 15 15:15:50 vm1 cib[14869]:  warning: retrieveCib: Cluster configuration not found: /var/lib/pacemaker/cib/cib.xml
Oct 15 15:15:50 vm1 cib[14869]:  warning: readCibXmlFile: Primary configuration corrupt or unusable, trying backups in /var/lib/pacemaker/cib
Oct 15 15:15:50 vm1 cib[14869]:  warning: readCibXmlFile: Continuing with an empty configuration.
Oct 15 15:15:50 vm1 cib[14869]:     info: validate_with_relaxng: Creating RNG parser context
Oct 15 15:15:50 vm1 attrd[14872]:   notice: crm_add_logfile: Additional logging available in /var/log/ha-debug
Oct 15 15:15:50 vm1 attrd[14872]:    debug: crm_update_callsites: Enabling callsites based on priority=7, files=(null), functions=(null), formats=(null), tags=(null)
Oct 15 15:15:50 vm1 attrd[14872]:     info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster
Oct 15 15:15:50 vm1 attrd[14872]:     info: main: Starting up
Oct 15 15:15:50 vm1 attrd[14872]:     info: get_cluster_type: Verifying cluster type: 'corosync'
Oct 15 15:15:50 vm1 attrd[14872]:     info: get_cluster_type: Assuming an active 'corosync' cluster
Oct 15 15:15:50 vm1 attrd[14872]:   notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync
Oct 15 15:15:50 vm1 lrmd[14871]:   notice: crm_add_logfile: Additional logging available in /var/log/ha-debug
Oct 15 15:15:50 vm1 lrmd[14871]:    debug: crm_update_callsites: Enabling callsites based on priority=7, files=(null), functions=(null), formats=(null), tags=(null)
Oct 15 15:15:50 vm1 lrmd[14871]:     info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/root
Oct 15 15:15:50 vm1 pengine[14873]:   notice: crm_add_logfile: Additional logging available in /var/log/ha-debug
Oct 15 15:15:50 vm1 pengine[14873]:    debug: crm_update_callsites: Enabling callsites based on priority=7, files=(null), functions=(null), formats=(null), tags=(null)
Oct 15 15:15:50 vm1 lrmd[14871]:     info: qb_ipcs_us_publish: server name: lrmd
Oct 15 15:15:50 vm1 lrmd[14871]:     info: main: Starting
Oct 15 15:15:50 vm1 pengine[14873]:     info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster
Oct 15 15:15:50 vm1 pengine[14873]:     info: qb_ipcs_us_publish: server name: pengine
Oct 15 15:15:50 vm1 pengine[14873]:     info: main: Starting pengine
Oct 15 15:15:50 vm1 crmd[14874]:   notice: crm_add_logfile: Additional logging available in /var/log/ha-debug
Oct 15 15:15:50 vm1 crmd[14874]:    debug: crm_update_callsites: Enabling callsites based on priority=7, files=(null), functions=(null), formats=(null), tags=(null)
Oct 15 15:15:50 vm1 crmd[14874]:     info: crm_log_init: Changed active directory to /var/lib/heartbeat/cores/hacluster
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261518
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: crm_update_peer_state: pcmk_quorum_notification: Node (null)[3232261518] - state is now member (was (null))
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_get_peer: Created entry a0bc9878-acc9-4510-bfd4-6416b4e4ddd2/0xa63530 for node (null)/3232261519 (3 total)
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_get_peer: Node 3232261519 has uuid 3232261519
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: pcmk_quorum_notification: Obtaining name for new node 3232261519
Oct 15 15:15:50 vm1 crmd[14874]:   notice: main: CRM Git Version: 6a5e863
Oct 15 15:15:50 vm1 crmd[14874]:     info: do_log: FSA: Input I_STARTUP from crmd_init() received in state S_STARTING
Oct 15 15:15:50 vm1 crmd[14874]:     info: get_cluster_type: Verifying cluster type: 'corosync'
Oct 15 15:15:50 vm1 crmd[14874]:     info: get_cluster_type: Assuming an active 'corosync' cluster
Oct 15 15:15:50 vm1 crmd[14874]:     info: crm_ipc_connect: Could not establish cib_shm connection: Connection refused (111)
Oct 15 15:15:50 vm1 cib[14869]:     info: startCib: CIB Initialization completed successfully
Oct 15 15:15:50 vm1 cib[14869]:   notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync
Oct 15 15:15:50 vm1 stonith-ng[14870]:     info: crm_get_peer: Created entry 9bc0ac17-61d3-4850-b65d-ec668a119d7d/0xd2c660 for node (null)/3232261517 (1 total)
Oct 15 15:15:50 vm1 stonith-ng[14870]:     info: crm_get_peer: Node 3232261517 has uuid 3232261517
Oct 15 15:15:50 vm1 stonith-ng[14870]:     info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3232261517] - corosync-cpg is now online
Oct 15 15:15:50 vm1 stonith-ng[14870]:     info: init_cs_connection_once: Connection to 'corosync': established
Oct 15 15:15:50 vm1 attrd[14872]:     info: crm_get_peer: Created entry 3643c6c1-537a-499b-95bb-5577c12b9937/0x1760120 for node (null)/3232261517 (1 total)
Oct 15 15:15:50 vm1 attrd[14872]:     info: crm_get_peer: Node 3232261517 has uuid 3232261517
Oct 15 15:15:50 vm1 attrd[14872]:     info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3232261517] - corosync-cpg is now online
Oct 15 15:15:50 vm1 attrd[14872]:   notice: crm_update_peer_state: attrd_peer_change_cb: Node (null)[3232261517] - state is now member (was (null))
Oct 15 15:15:50 vm1 attrd[14872]:     info: init_cs_connection_once: Connection to 'corosync': established
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261519
Oct 15 15:15:50 vm1 pacemakerd[14865]:   notice: crm_update_peer_state: pcmk_quorum_notification: Node (null)[3232261519] - state is now member (was (null))
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_get_peer: Node 3232261518 is now known as vm2
Oct 15 15:15:50 vm1 pacemakerd[14865]:     info: crm_get_peer: Node 3232261519 is now known as vm3
Oct 15 15:15:50 vm1 stonith-ng[14870]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261517
Oct 15 15:15:50 vm1 stonith-ng[14870]:   notice: get_node_name: Defaulting to uname -n for the local corosync node name
Oct 15 15:15:50 vm1 stonith-ng[14870]:     info: crm_get_peer: Node 3232261517 is now known as vm1
Oct 15 15:15:50 vm1 stonith-ng[14870]:     info: crm_ipc_connect: Could not establish cib_rw connection: Connection refused (111)
Oct 15 15:15:50 vm1 cib[14869]:     info: crm_get_peer: Created entry 625a1694-8174-49bc-8f1b-1f96d5a5319c/0x122c0b0 for node (null)/3232261517 (1 total)
Oct 15 15:15:50 vm1 cib[14869]:     info: crm_get_peer: Node 3232261517 has uuid 3232261517
Oct 15 15:15:50 vm1 cib[14869]:     info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3232261517] - corosync-cpg is now online
Oct 15 15:15:50 vm1 cib[14869]:     info: init_cs_connection_once: Connection to 'corosync': established
Oct 15 15:15:50 vm1 attrd[14872]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261517
Oct 15 15:15:50 vm1 attrd[14872]:   notice: get_node_name: Defaulting to uname -n for the local corosync node name
Oct 15 15:15:50 vm1 attrd[14872]:     info: crm_get_peer: Node 3232261517 is now known as vm1
Oct 15 15:15:50 vm1 attrd[14872]:     info: main: Cluster connection active
Oct 15 15:15:50 vm1 attrd[14872]:     info: qb_ipcs_us_publish: server name: attrd
Oct 15 15:15:50 vm1 attrd[14872]:     info: main: Accepting attribute updates
Oct 15 15:15:50 vm1 attrd[14872]:     info: crm_ipc_connect: Could not establish cib_rw connection: Connection refused (111)
Oct 15 15:15:50 vm1 cib[14869]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261517
Oct 15 15:15:50 vm1 cib[14869]:   notice: get_node_name: Defaulting to uname -n for the local corosync node name
Oct 15 15:15:50 vm1 cib[14869]:     info: crm_get_peer: Node 3232261517 is now known as vm1
Oct 15 15:15:50 vm1 cib[14869]:     info: qb_ipcs_us_publish: server name: cib_ro
Oct 15 15:15:50 vm1 cib[14869]:     info: qb_ipcs_us_publish: server name: cib_rw
Oct 15 15:15:50 vm1 cib[14869]:     info: qb_ipcs_us_publish: server name: cib_shm
Oct 15 15:15:50 vm1 cib[14869]:     info: cib_init: Starting cib mainloop
Oct 15 15:15:50 vm1 cib[14869]:     info: pcmk_cpg_membership: Joined[0.0] cib.3232261517 
Oct 15 15:15:50 vm1 cib[14869]:     info: pcmk_cpg_membership: Member[0.0] cib.3232261517 
Oct 15 15:15:50 vm1 cib[14869]:     info: crm_get_peer: Created entry 051cade5-4f39-41a1-bc91-3104583927bd/0x122e940 for node (null)/3232261518 (2 total)
Oct 15 15:15:50 vm1 cib[14869]:     info: crm_get_peer: Node 3232261518 has uuid 3232261518
Oct 15 15:15:50 vm1 cib[14869]:     info: pcmk_cpg_membership: Member[0.1] cib.3232261518 
Oct 15 15:15:50 vm1 cib[14869]:     info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[3232261518] - corosync-cpg is now online
Oct 15 15:15:50 vm1 cib[14869]:     info: pcmk_cpg_membership: Joined[1.0] cib.3232261519 
Oct 15 15:15:50 vm1 cib[14869]:     info: pcmk_cpg_membership: Member[1.0] cib.3232261517 
Oct 15 15:15:50 vm1 cib[14869]:     info: pcmk_cpg_membership: Member[1.1] cib.3232261518 
Oct 15 15:15:50 vm1 cib[14869]:     info: crm_get_peer: Created entry c0fdc043-f03f-41c0-a0e3-7d454eeaed0d/0x122e9b0 for node (null)/3232261519 (3 total)
Oct 15 15:15:50 vm1 cib[14869]:     info: crm_get_peer: Node 3232261519 has uuid 3232261519
Oct 15 15:15:50 vm1 cib[14869]:     info: pcmk_cpg_membership: Member[1.2] cib.3232261519 
Oct 15 15:15:50 vm1 cib[14869]:     info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[3232261519] - corosync-cpg is now online
Oct 15 15:15:50 vm1 cib[14875]:     info: write_cib_contents: Wrote version 0.0.0 of the CIB to disk (digest: 3930c46445d2289a49a22e68ead11aaf)
Oct 15 15:15:50 vm1 cib[14875]:     info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.za5CTj (digest: /var/lib/pacemaker/cib/cib.nBTEUu)
Oct 15 15:15:51 vm1 cib[14869]:     info: crm_client_new: Connecting 0x122f110 for uid=0 gid=0 pid=23331 id=06d6cc80-cc71-4fe3-8dc1-0b5a2aeb1a69
Oct 15 15:15:51 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crm_mon/29, version=0.0.0)
Oct 15 15:15:51 vm1 cib[14869]:     info: crm_client_new: Connecting 0x12b3320 for uid=496 gid=492 pid=14874 id=5b415466-b331-49a1-b495-ed97c3cb8b21
Oct 15 15:15:51 vm1 crmd[14874]:     info: do_cib_control: CIB connection established
Oct 15 15:15:51 vm1 crmd[14874]:   notice: crm_cluster_connect: Connecting to cluster infrastructure: corosync
Oct 15 15:15:51 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/2, version=0.0.0)
Oct 15 15:15:51 vm1 crmd[14874]:     info: crm_get_peer: Created entry 588c7fd2-4783-4439-b2b1-192c7b6bd1f4/0x16e1ec0 for node (null)/3232261517 (1 total)
Oct 15 15:15:51 vm1 crmd[14874]:     info: crm_get_peer: Node 3232261517 has uuid 3232261517
Oct 15 15:15:51 vm1 crmd[14874]:     info: crm_update_peer_proc: cluster_connect_cpg: Node (null)[3232261517] - corosync-cpg is now online
Oct 15 15:15:51 vm1 crmd[14874]:     info: init_cs_connection_once: Connection to 'corosync': established
Oct 15 15:15:51 vm1 crmd[14874]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261517
Oct 15 15:15:51 vm1 crmd[14874]:   notice: get_node_name: Defaulting to uname -n for the local corosync node name
Oct 15 15:15:51 vm1 crmd[14874]:     info: crm_get_peer: Node 3232261517 is now known as vm1
Oct 15 15:15:51 vm1 crmd[14874]:     info: peer_update_callback: vm1 is now (null)
Oct 15 15:15:51 vm1 crmd[14874]:   notice: cluster_connect_quorum: Quorum acquired
Oct 15 15:15:51 vm1 cib[14869]:     info: crm_client_new: Connecting 0x12b1a10 for uid=0 gid=0 pid=14870 id=d66f721c-1ad2-4350-9a72-db8b684f58c2
Oct 15 15:15:51 vm1 stonith-ng[14870]:   notice: setup_cib: Watching for stonith topology changes
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: qb_ipcs_us_publish: server name: stonith-ng
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: main: Starting stonith-ng mainloop
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: pcmk_cpg_membership: Joined[0.0] stonith-ng.3232261517 
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: pcmk_cpg_membership: Member[0.0] stonith-ng.3232261517 
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: crm_get_peer: Created entry e8e88268-fab1-4a08-8681-3587011ea75e/0xd30690 for node (null)/3232261518 (2 total)
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: crm_get_peer: Node 3232261518 has uuid 3232261518
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: pcmk_cpg_membership: Member[0.1] stonith-ng.3232261518 
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[3232261518] - corosync-cpg is now online
Oct 15 15:15:51 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/2, version=0.0.0)
Oct 15 15:15:51 vm1 crmd[14874]:     info: do_ha_control: Connected to the cluster
Oct 15 15:15:51 vm1 crmd[14874]:     info: lrmd_ipc_connect: Connecting to lrmd
Oct 15 15:15:51 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=local/crmd/3, version=0.0.0)
Oct 15 15:15:51 vm1 lrmd[14871]:     info: crm_client_new: Connecting 0xef8d10 for uid=496 gid=492 pid=14874 id=0245b4d8-632b-498b-abda-91d20df4709f
Oct 15 15:15:51 vm1 cib[14869]:     info: crm_client_new: Connecting 0x12b5af0 for uid=496 gid=492 pid=14872 id=9176b503-a83c-4b85-ba02-759e6090e522
Oct 15 15:15:51 vm1 crmd[14874]:     info: do_lrm_control: LRM connection established
Oct 15 15:15:51 vm1 crmd[14874]:     info: do_started: Delaying start, no membership data (0000000000100000)
Oct 15 15:15:51 vm1 crmd[14874]:     info: pcmk_quorum_notification: Membership 12: quorum retained (3)
Oct 15 15:15:51 vm1 crmd[14874]:   notice: crm_update_peer_state: pcmk_quorum_notification: Node vm1[3232261517] - state is now member (was (null))
Oct 15 15:15:51 vm1 crmd[14874]:     info: peer_update_callback: vm1 is now member (was (null))
Oct 15 15:15:51 vm1 crmd[14874]:     info: crm_get_peer: Created entry 372e060a-eee2-47a1-9239-6704e98945eb/0x1827970 for node (null)/3232261518 (2 total)
Oct 15 15:15:51 vm1 crmd[14874]:     info: crm_get_peer: Node 3232261518 has uuid 3232261518
Oct 15 15:15:51 vm1 crmd[14874]:     info: pcmk_quorum_notification: Obtaining name for new node 3232261518
Oct 15 15:15:51 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section crm_config: OK (rc=0, origin=local/crmd/4, version=0.0.0)
Oct 15 15:15:51 vm1 attrd[14872]:     info: attrd_cib_connect: Connected to the CIB after 2 attempts
Oct 15 15:15:51 vm1 attrd[14872]:     info: main: CIB connection active
Oct 15 15:15:51 vm1 attrd[14872]:     info: pcmk_cpg_membership: Joined[0.0] attrd.3232261517 
Oct 15 15:15:51 vm1 attrd[14872]:     info: pcmk_cpg_membership: Member[0.0] attrd.3232261517 
Oct 15 15:15:51 vm1 attrd[14872]:     info: crm_get_peer: Created entry abd2abca-d2be-43fe-bcc9-3a93e950db08/0x1765f60 for node (null)/3232261518 (2 total)
Oct 15 15:15:51 vm1 attrd[14872]:     info: crm_get_peer: Node 3232261518 has uuid 3232261518
Oct 15 15:15:51 vm1 attrd[14872]:     info: pcmk_cpg_membership: Member[0.1] attrd.3232261518 
Oct 15 15:15:51 vm1 attrd[14872]:     info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[3232261518] - corosync-cpg is now online
Oct 15 15:15:51 vm1 attrd[14872]:   notice: crm_update_peer_state: attrd_peer_change_cb: Node (null)[3232261518] - state is now member (was (null))
Oct 15 15:15:51 vm1 attrd[14872]:     info: pcmk_cpg_membership: Joined[1.0] attrd.3232261519 
Oct 15 15:15:51 vm1 attrd[14872]:     info: pcmk_cpg_membership: Member[1.0] attrd.3232261517 
Oct 15 15:15:51 vm1 attrd[14872]:     info: pcmk_cpg_membership: Member[1.1] attrd.3232261518 
Oct 15 15:15:51 vm1 attrd[14872]:     info: crm_get_peer: Created entry 7f3f4fd0-0841-42fc-af5b-e6433194a597/0x1765fd0 for node (null)/3232261519 (3 total)
Oct 15 15:15:51 vm1 attrd[14872]:     info: crm_get_peer: Node 3232261519 has uuid 3232261519
Oct 15 15:15:51 vm1 attrd[14872]:     info: pcmk_cpg_membership: Member[1.2] attrd.3232261519 
Oct 15 15:15:51 vm1 attrd[14872]:     info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[3232261519] - corosync-cpg is now online
Oct 15 15:15:51 vm1 attrd[14872]:   notice: crm_update_peer_state: attrd_peer_change_cb: Node (null)[3232261519] - state is now member (was (null))
Oct 15 15:15:51 vm1 stonith-ng[14870]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261517
Oct 15 15:15:51 vm1 stonith-ng[14870]:   notice: get_node_name: Defaulting to uname -n for the local corosync node name
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: pcmk_cpg_membership: Joined[1.0] stonith-ng.3232261519 
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: pcmk_cpg_membership: Member[1.0] stonith-ng.3232261517 
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: pcmk_cpg_membership: Member[1.1] stonith-ng.3232261518 
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: crm_get_peer: Created entry 1e858d23-45f8-43d5-8072-a64ea95bfbc0/0xd2e8e0 for node (null)/3232261519 (3 total)
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: crm_get_peer: Node 3232261519 has uuid 3232261519
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: pcmk_cpg_membership: Member[1.2] stonith-ng.3232261519 
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[3232261519] - corosync-cpg is now online
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: crm_get_peer: Node 3232261518 is now known as vm2
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: init_cib_cache_cb: Updating device list from the cib: init
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: unpack_nodes: Creating a fake local node
Oct 15 15:15:51 vm1 crmd[14874]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261518
Oct 15 15:15:51 vm1 crmd[14874]:   notice: crm_update_peer_state: pcmk_quorum_notification: Node (null)[3232261518] - state is now member (was (null))
Oct 15 15:15:51 vm1 crmd[14874]:     info: crm_get_peer: Created entry 40021629-1033-4f01-9550-f1b5b8ecbf3f/0x1827860 for node (null)/3232261519 (3 total)
Oct 15 15:15:51 vm1 crmd[14874]:     info: crm_get_peer: Node 3232261519 has uuid 3232261519
Oct 15 15:15:51 vm1 crmd[14874]:     info: pcmk_quorum_notification: Obtaining name for new node 3232261519
Oct 15 15:15:51 vm1 crmd[14874]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261519
Oct 15 15:15:51 vm1 crmd[14874]:   notice: crm_update_peer_state: pcmk_quorum_notification: Node (null)[3232261519] - state is now member (was (null))
Oct 15 15:15:51 vm1 stonith-ng[14870]:     info: crm_get_peer: Node 3232261519 is now known as vm3
Oct 15 15:15:51 vm1 crmd[14874]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261517
Oct 15 15:15:51 vm1 crmd[14874]:   notice: get_node_name: Defaulting to uname -n for the local corosync node name
Oct 15 15:15:51 vm1 crmd[14874]:     info: do_started: Delaying start, Config not read (0000000000000040)
Oct 15 15:15:51 vm1 crmd[14874]:     info: qb_ipcs_us_publish: server name: crmd
Oct 15 15:15:51 vm1 crmd[14874]:   notice: do_started: The local CRM is operational
Oct 15 15:15:51 vm1 crmd[14874]:     info: do_log: FSA: Input I_PENDING from do_started() received in state S_STARTING
Oct 15 15:15:51 vm1 crmd[14874]:   notice: do_state_transition: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_started ]
Oct 15 15:15:51 vm1 cib[14869]:     info: cib_process_request: Completed cib_slave operation for section 'all': OK (rc=0, origin=local/crmd/5, version=0.0.0)
Oct 15 15:15:52 vm1 crmd[14874]:     info: pcmk_cpg_membership: Joined[0.0] crmd.3232261517 
Oct 15 15:15:52 vm1 crmd[14874]:     info: pcmk_cpg_membership: Member[0.0] crmd.3232261517 
Oct 15 15:15:52 vm1 crmd[14874]:     info: pcmk_cpg_membership: Member[0.1] crmd.3232261518 
Oct 15 15:15:52 vm1 crmd[14874]:     info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[3232261518] - corosync-cpg is now online
Oct 15 15:15:52 vm1 crmd[14874]:     info: crm_get_peer: Node 3232261518 is now known as vm2
Oct 15 15:15:52 vm1 crmd[14874]:     info: peer_update_callback: vm2 is now member
Oct 15 15:15:52 vm1 crmd[14874]:     info: pcmk_cpg_membership: Joined[1.0] crmd.3232261519 
Oct 15 15:15:52 vm1 crmd[14874]:     info: pcmk_cpg_membership: Member[1.0] crmd.3232261517 
Oct 15 15:15:52 vm1 crmd[14874]:     info: pcmk_cpg_membership: Member[1.1] crmd.3232261518 
Oct 15 15:15:52 vm1 crmd[14874]:     info: pcmk_cpg_membership: Member[1.2] crmd.3232261519 
Oct 15 15:15:52 vm1 crmd[14874]:     info: crm_update_peer_proc: pcmk_cpg_membership: Node (null)[3232261519] - corosync-cpg is now online
Oct 15 15:15:52 vm1 crmd[14874]:     info: crm_get_peer: Node 3232261519 is now known as vm3
Oct 15 15:15:52 vm1 crmd[14874]:     info: peer_update_callback: vm3 is now member
Oct 15 15:15:53 vm1 stonith-ng[14870]:     info: crm_client_new: Connecting 0xd37980 for uid=496 gid=492 pid=14874 id=4eb0ff33-6154-4b90-9801-dc2d005b765a
Oct 15 15:15:53 vm1 stonith-ng[14870]:     info: stonith_command: Processed register from crmd.14874: OK (0)
Oct 15 15:15:53 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_notify from crmd.14874: OK (0)
Oct 15 15:15:53 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_notify from crmd.14874: OK (0)
Oct 15 15:16:12 vm1 crmd[14874]:     info: election_count_vote: Election 1 (owner: 3232261518) pass: vote from vm2 (Uptime)
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_state_transition: State transition S_PENDING -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Oct 15 15:16:12 vm1 crmd[14874]:     info: election_count_vote: Election 1 (owner: 3232261519) pass: vote from vm3 (Uptime)
Oct 15 15:16:12 vm1 crmd[14874]:     info: election_timer_cb: Election election-0 complete
Oct 15 15:16:12 vm1 crmd[14874]:     info: election_timeout_popped: Election failed: Declaring ourselves the winner
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_log: FSA: Input I_ELECTION_DC from election_timeout_popped() received in state S_ELECTION
Oct 15 15:16:12 vm1 crmd[14874]:   notice: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_TIMER_POPPED origin=election_timeout_popped ]
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_te_control: Registering TE UUID: cffe5b98-3c92-4ed3-8992-426ef00df4ed
Oct 15 15:16:12 vm1 crmd[14874]:     info: set_graph_functions: Setting custom graph functions
Oct 15 15:16:12 vm1 pengine[14873]:     info: crm_client_new: Connecting 0x196a700 for uid=496 gid=492 pid=14874 id=8686c625-81d6-4dec-a38e-7e834e95c904
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_dc_takeover: Taking over DC status for this partition
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_readwrite: We are now in R/W mode
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/6, version=0.0.0)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section cib: OK (rc=0, origin=local/crmd/7, version=0.0.1)
Oct 15 15:16:12 vm1 cib[14869]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261517
Oct 15 15:16:12 vm1 cib[14869]:   notice: get_node_name: Defaulting to uname -n for the local corosync node name
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section //cib/configuration/crm_config//cluster_property_set//nvpair[@name='dc-version']: No such device or address (rc=-6, origin=local/crmd/8, version=0.0.1)
Oct 15 15:16:12 vm1 cib[14869]:   notice: log_cib_diff: cib:diff: Local-only Change: 0.1.1
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: -- <cib admin_epoch="0" epoch="0" num_updates="1"/>
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: ++       <cluster_property_set id="cib-bootstrap-options">
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: ++         <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.11-0.284.6a5e863.git.el6-6a5e863"/>
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: ++       </cluster_property_set>
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section crm_config: OK (rc=0, origin=local/crmd/9, version=0.1.1)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section //cib/configuration/crm_config//cluster_property_set//nvpair[@name='cluster-infrastructure']: No such device or address (rc=-6, origin=local/crmd/10, version=0.1.1)
Oct 15 15:16:12 vm1 crmd[14874]:     info: join_make_offer: Making join offers based on membership 12
Oct 15 15:16:12 vm1 crmd[14874]:     info: join_make_offer: join-1: Sending offer to vm3
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm3[3232261519] - join-1 phase 0 -> 1
Oct 15 15:16:12 vm1 crmd[14874]:     info: join_make_offer: join-1: Sending offer to vm1
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm1[3232261517] - join-1 phase 0 -> 1
Oct 15 15:16:12 vm1 crmd[14874]:     info: join_make_offer: join-1: Sending offer to vm2
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm2[3232261518] - join-1 phase 0 -> 1
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_dc_join_offer_all: join-1: Waiting on 3 outstanding join acks
Oct 15 15:16:12 vm1 crmd[14874]:  warning: do_log: FSA: Input I_ELECTION_DC from do_election_check() received in state S_INTEGRATION
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: initialize_join: Node vm3[3232261519] - join-2 phase 1 -> 0
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: initialize_join: Node vm1[3232261517] - join-2 phase 1 -> 0
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: initialize_join: Node vm2[3232261518] - join-2 phase 1 -> 0
Oct 15 15:16:12 vm1 crmd[14874]:     info: join_make_offer: join-2: Sending offer to vm3
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm3[3232261519] - join-2 phase 0 -> 1
Oct 15 15:16:12 vm1 crmd[14874]:     info: join_make_offer: join-2: Sending offer to vm1
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm1[3232261517] - join-2 phase 0 -> 1
Oct 15 15:16:12 vm1 crmd[14874]:     info: join_make_offer: join-2: Sending offer to vm2
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm2[3232261518] - join-2 phase 0 -> 1
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_dc_join_offer_all: join-2: Waiting on 3 outstanding join acks
Oct 15 15:16:12 vm1 cib[14869]:   notice: log_cib_diff: cib:diff: Local-only Change: 0.2.1
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: -- <cib admin_epoch="0" epoch="1" num_updates="1"/>
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: ++         <nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/>
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section crm_config: OK (rc=0, origin=local/crmd/11, version=0.2.1)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section crm_config: OK (rc=0, origin=local/crmd/12, version=0.2.1)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section crm_config: OK (rc=0, origin=local/crmd/13, version=0.2.1)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section crm_config: OK (rc=0, origin=local/crmd/14, version=0.2.1)
Oct 15 15:16:12 vm1 cib[14876]:     info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-0.raw
Oct 15 15:16:12 vm1 crmd[14874]:     info: update_dc: Set DC to vm1 (3.0.7)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/15, version=0.2.1)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/16, version=0.2.1)
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_filter_offer: Node vm1[3232261517] - join-2 phase 1 -> 2
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_expected: do_dc_join_filter_offer: Node vm1[3232261517] - expected state is now member
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_filter_offer: Node vm2[3232261518] - join-2 phase 1 -> 2
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_expected: do_dc_join_filter_offer: Node vm2[3232261518] - expected state is now member
Oct 15 15:16:12 vm1 cib[14876]:     info: write_cib_contents: Wrote version 0.1.0 of the CIB to disk (digest: 51521f2153f57cb386e10b5d3317b80b)
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_filter_offer: Node vm3[3232261519] - join-2 phase 1 -> 2
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_expected: do_dc_join_filter_offer: Node vm3[3232261519] - expected state is now member
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Oct 15 15:16:12 vm1 crmd[14874]:     info: crmd_join_phase_log: join-2: vm3=integrated
Oct 15 15:16:12 vm1 crmd[14874]:     info: crmd_join_phase_log: join-2: vm1=integrated
Oct 15 15:16:12 vm1 crmd[14874]:     info: crmd_join_phase_log: join-2: vm2=integrated
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_dc_join_finalize: join-2: Syncing our CIB to the rest of the cluster
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_sync operation for section 'all': OK (rc=0, origin=local/crmd/17, version=0.2.1)
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: finalize_join_for: Node vm3[3232261519] - join-2 phase 2 -> 3
Oct 15 15:16:12 vm1 cib[14876]:     info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.1mdLng (digest: /var/lib/pacemaker/cib/cib.RPb3up)
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: finalize_join_for: Node vm1[3232261517] - join-2 phase 2 -> 3
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: finalize_join_for: Node vm2[3232261518] - join-2 phase 2 -> 3
Oct 15 15:16:12 vm1 crmd[14874]:     info: erase_status_tag: Deleting xpath: //node_state[@uname='vm1']/transient_attributes
Oct 15 15:16:12 vm1 crmd[14874]:     info: update_attrd_helper: Connecting to attrd... 5 retries remaining
Oct 15 15:16:12 vm1 cib[14869]:   notice: log_cib_diff: cib:diff: Local-only Change: 0.3.1
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: -- <cib admin_epoch="0" epoch="2" num_updates="1"/>
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: ++       <node id="3232261519" uname="vm3"/>
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=local/crmd/18, version=0.3.1)
Oct 15 15:16:12 vm1 attrd[14872]:     info: crm_client_new: Connecting 0x1763460 for uid=496 gid=492 pid=14874 id=1810848f-23b4-48a4-8b29-8fcd652ba5be
Oct 15 15:16:12 vm1 cib[14869]:   notice: log_cib_diff: cib:diff: Local-only Change: 0.4.1
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: -- <cib admin_epoch="0" epoch="3" num_updates="1"/>
Oct 15 15:16:12 vm1 attrd[14872]:     info: attrd_client_message: Starting an election to determine the writer
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_ack: Node vm2[3232261518] - join-2 phase 3 -> 4
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_dc_join_ack: join-2: Updating node state to member for vm2
Oct 15 15:16:12 vm1 crmd[14874]:     info: erase_status_tag: Deleting xpath: //node_state[@uname='vm2']/lrm
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_ack: Node vm3[3232261519] - join-2 phase 3 -> 4
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_dc_join_ack: join-2: Updating node state to member for vm3
Oct 15 15:16:12 vm1 crmd[14874]:     info: erase_status_tag: Deleting xpath: //node_state[@uname='vm3']/lrm
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: ++       <node id="3232261517" uname="vm1"/>
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=local/crmd/19, version=0.4.1)
Oct 15 15:16:12 vm1 cib[14869]:   notice: log_cib_diff: cib:diff: Local-only Change: 0.5.1
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: -- <cib admin_epoch="0" epoch="4" num_updates="1"/>
Oct 15 15:16:12 vm1 cib[14869]:   notice: cib:diff: ++       <node id="3232261518" uname="vm2"/>
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=local/crmd/20, version=0.5.1)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='vm1']/transient_attributes: OK (rc=0, origin=local/crmd/21, version=0.5.1)
Oct 15 15:16:12 vm1 cib[14869]:     info: crm_get_peer: Node 3232261518 is now known as vm2
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='vm2']/transient_attributes: OK (rc=0, origin=vm2/crmd/9, version=0.5.1)
Oct 15 15:16:12 vm1 cib[14869]:     info: crm_get_peer: Node 3232261519 is now known as vm3
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='vm3']/transient_attributes: OK (rc=0, origin=vm3/crmd/9, version=0.5.1)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='vm2']/lrm: OK (rc=0, origin=local/crmd/22, version=0.5.1)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/23, version=0.5.2)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='vm3']/lrm: OK (rc=0, origin=local/crmd/24, version=0.5.2)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/25, version=0.5.3)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: corosync_node_name: Unable to get node name for nodeid 3232261517
Oct 15 15:16:12 vm1 attrd[14872]:   notice: get_node_name: Defaulting to uname -n for the local corosync node name
Oct 15 15:16:12 vm1 attrd[14872]:     info: attrd_client_message: Broadcasting terminate[vm1] = (null)
Oct 15 15:16:12 vm1 attrd[14872]:     info: attrd_client_message: Broadcasting shutdown[vm1] = (null)
Oct 15 15:16:12 vm1 attrd[14872]:     info: crm_get_peer: Node 3232261519 is now known as vm3
Oct 15 15:16:12 vm1 attrd[14872]:     info: election_count_vote: Election 1 (owner: 3232261519) pass: vote from vm3 (Uptime)
Oct 15 15:16:12 vm1 cib[14877]:     info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-1.raw
Oct 15 15:16:12 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_ack: Node vm1[3232261517] - join-2 phase 3 -> 4
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_dc_join_ack: join-2: Updating node state to member for vm1
Oct 15 15:16:12 vm1 crmd[14874]:     info: erase_status_tag: Deleting xpath: //node_state[@uname='vm1']/lrm
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='vm1']/lrm: OK (rc=0, origin=local/crmd/26, version=0.5.3)
Oct 15 15:16:12 vm1 attrd[14872]:     info: crm_get_peer: Node 3232261518 is now known as vm2
Oct 15 15:16:12 vm1 attrd[14872]:     info: election_count_vote: Election 1 (owner: 3232261518) pass: vote from vm2 (Uptime)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/27, version=0.5.4)
Oct 15 15:16:12 vm1 attrd[14872]:     info: election_count_vote: Election 2 (owner: 3232261518) pass: vote from vm2 (Uptime)
Oct 15 15:16:12 vm1 cib[14877]:     info: write_cib_contents: Wrote version 0.5.0 of the CIB to disk (digest: 034de6ab489c499a76b66b8122cf302a)
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Oct 15 15:16:12 vm1 crmd[14874]:     info: abort_transition_graph: do_te_invoke:151 - Triggered transition abort (complete=1) : Peer Cancelled
Oct 15 15:16:12 vm1 cib[14877]:     info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.A52rgl (digest: /var/lib/pacemaker/cib/cib.I86ryu)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=local/crmd/28, version=0.5.4)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/29, version=0.5.4)
Oct 15 15:16:12 vm1 attrd[14872]:     info: election_timer_cb: Election election-attrd complete
Oct 15 15:16:12 vm1 attrd[14872]:   notice: write_attribute: Sent update 2 with 3 changes for shutdown, id=<n/a>, set=(null)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: write_attribute: Sent update 3 with 3 changes for terminate, id=<n/a>, set=(null)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section cib: OK (rc=0, origin=local/crmd/30, version=0.5.5)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/31, version=0.5.5)
Oct 15 15:16:12 vm1 pengine[14873]:    error: unpack_resources: Resource start-up disabled since no STONITH resources have been defined
Oct 15 15:16:12 vm1 pengine[14873]:    error: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option
Oct 15 15:16:12 vm1 pengine[14873]:    error: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity
Oct 15 15:16:12 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm2 is active
Oct 15 15:16:12 vm1 pengine[14873]:     info: determine_online_status: Node vm2 is online
Oct 15 15:16:12 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm3 is active
Oct 15 15:16:12 vm1 pengine[14873]:     info: determine_online_status: Node vm3 is online
Oct 15 15:16:12 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm1 is active
Oct 15 15:16:12 vm1 pengine[14873]:     info: determine_online_status: Node vm1 is online
Oct 15 15:16:12 vm1 pengine[14873]:   notice: stage6: Delaying fencing operations until there are resources to manage
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/attrd/2, version=0.5.6)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/attrd/3, version=0.5.6)
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_te_invoke: Processing graph 0 (ref=pe_calc-dc-1381817772-16) derived from /var/lib/pacemaker/pengine/pe-input-0.bz2
Oct 15 15:16:12 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 4: probe_complete probe_complete on vm3 - no waiting
Oct 15 15:16:12 vm1 crmd[14874]:     info: te_rsc_command: Action 4 confirmed - no wait
Oct 15 15:16:12 vm1 pengine[14873]:   notice: process_pe_message: Calculated Transition 0: /var/lib/pacemaker/pengine/pe-input-0.bz2
Oct 15 15:16:12 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 3: probe_complete probe_complete on vm2 - no waiting
Oct 15 15:16:12 vm1 pengine[14873]:   notice: process_pe_message: Configuration ERRORs found during PE processing.  Please run "crm_verify -L" to identify issues.
Oct 15 15:16:12 vm1 crmd[14874]:     info: te_rsc_command: Action 3 confirmed - no wait
Oct 15 15:16:12 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 2: probe_complete probe_complete on vm1 (local) - no waiting
Oct 15 15:16:12 vm1 crmd[14874]:     info: te_rsc_command: Action 2 confirmed - no wait
Oct 15 15:16:12 vm1 attrd[14872]:     info: attrd_client_message: Broadcasting probe_complete[vm1] = true (writer)
Oct 15 15:16:12 vm1 crmd[14874]:   notice: run_graph: Transition 0 (Complete=3, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-0.bz2): Complete
Oct 15 15:16:12 vm1 attrd[14872]:     info: attrd_cib_callback: Update 3 for terminate: OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 3 for terminate[vm1]=(null): OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 3 for terminate[vm2]=(null): OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 3 for terminate[vm3]=(null): OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:     info: attrd_cib_callback: Update 2 for shutdown: OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 2 for shutdown[vm1]=(null): OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 2 for shutdown[vm2]=(null): OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 2 for shutdown[vm3]=(null): OK (0)
Oct 15 15:16:12 vm1 crmd[14874]:     info: do_log: FSA: Input I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE
Oct 15 15:16:12 vm1 crmd[14874]:   notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 15 15:16:12 vm1 attrd[14872]:   notice: write_attribute: Sent update 4 with 1 changes for probe_complete, id=<n/a>, set=(null)
Oct 15 15:16:12 vm1 attrd[14872]:     info: write_attribute: Write out of probe_complete delayed: update 4 in progress
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/attrd/4, version=0.5.7)
Oct 15 15:16:12 vm1 attrd[14872]:     info: write_attribute: Write out of probe_complete delayed: update 4 in progress
Oct 15 15:16:12 vm1 attrd[14872]:     info: attrd_cib_callback: Update 4 for probe_complete: OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 4 for probe_complete[vm1]=true: OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 4 for probe_complete[vm2]=(null): OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 4 for probe_complete[vm3]=(null): OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: write_attribute: Sent update 5 with 3 changes for probe_complete, id=<n/a>, set=(null)
Oct 15 15:16:12 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/attrd/5, version=0.5.8)
Oct 15 15:16:12 vm1 attrd[14872]:     info: attrd_cib_callback: Update 5 for probe_complete: OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 5 for probe_complete[vm1]=true: OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 5 for probe_complete[vm2]=true: OK (0)
Oct 15 15:16:12 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 5 for probe_complete[vm3]=true: OK (0)
Oct 15 15:16:21 vm1 cib[14869]:     info: crm_client_new: Connecting 0x1441290 for uid=0 gid=0 pid=14881 id=a77c8b98-a009-4d98-8874-7a6469166f7e
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/cibadmin/2, version=0.5.8)
Oct 15 15:16:21 vm1 cib[14869]:     info: crm_client_destroy: Destroying 0 events
Oct 15 15:16:21 vm1 cib[14869]:     info: crm_client_new: Connecting 0x1441290 for uid=0 gid=0 pid=14882 id=d36183bc-0a1f-4e87-9823-8b6bf77863af
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/cibadmin/2, version=0.5.8)
Oct 15 15:16:21 vm1 cib[14869]:     info: crm_client_destroy: Destroying 0 events
Oct 15 15:16:21 vm1 cib[14869]:     info: crm_client_new: Connecting 0x1441290 for uid=0 gid=0 pid=14922 id=83eacdc1-8fdf-496d-9a1f-06a9a962d7bd
Oct 15 15:16:21 vm1 crmd[14874]:     info: abort_transition_graph: te_update_diff:126 - Triggered transition abort (complete=1, node=, tag=diff, id=(null), magic=NA, cib=0.6.1) : Non-status change
Oct 15 15:16:21 vm1 crmd[14874]:   notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Oct 15 15:16:21 vm1 stonith-ng[14870]:     info: stonith_level_remove: Node vm3 not found (0 active entries)
Oct 15 15:16:21 vm1 stonith-ng[14870]:     info: stonith_level_register: Node vm3 has 1 active fencing levels
Oct 15 15:16:21 vm1 stonith-ng[14870]:     info: stonith_level_register: Node vm3 has 2 active fencing levels
Oct 15 15:16:21 vm1 stonith-ng[14870]:     info: update_cib_stonith_devices: Updating device list from the cib: new resource
Oct 15 15:16:21 vm1 stonith-ng[14870]:  warning: handle_startup_fencing: Blind faith: not fencing unseen nodes
Oct 15 15:16:21 vm1 stonith-ng[14870]:     info: cib_device_update: Device f1 is allowed on vm1: score=200
Oct 15 15:16:21 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action metadata for agent fence_legacy (target=(null))
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_replace_notify: Replaced: 0.5.8 -> 0.6.1 from vm1
Oct 15 15:16:21 vm1 crmd[14874]:     info: do_state_transition: State transition S_POLICY_ENGINE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL origin=do_cib_replaced ]
Oct 15 15:16:21 vm1 crmd[14874]:     info: update_dc: Unset DC. Was vm1
Oct 15 15:16:21 vm1 attrd[14872]:   notice: attrd_cib_replaced_cb: Updating all attributes after cib_refresh_notify event
Oct 15 15:16:21 vm1 attrd[14872]:   notice: write_attribute: Sent update 6 with 3 changes for shutdown, id=<n/a>, set=(null)
Oct 15 15:16:21 vm1 attrd[14872]:   notice: write_attribute: Sent update 7 with 3 changes for terminate, id=<n/a>, set=(null)
Oct 15 15:16:21 vm1 attrd[14872]:   notice: write_attribute: Sent update 8 with 3 changes for probe_complete, id=<n/a>, set=(null)
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: Diff: --- 0.5.8
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: Diff: +++ 0.6.1 1c8a43265f13cefc2951afe21b89b052
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: --         <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.11-0.284.6a5e863.git.el6-6a5e863"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: --         <nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <nvpair name="no-quorum-policy" value="freeze" id="cib-bootstrap-options-no-quorum-policy"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <nvpair name="stonith-enabled" value="true" id="cib-bootstrap-options-stonith-enabled"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <nvpair name="startup-fencing" value="false" id="cib-bootstrap-options-startup-fencing"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <nvpair name="stonith-timeout" value="60s" id="cib-bootstrap-options-stonith-timeout"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <nvpair name="crmd-transition-delay" value="2s" id="cib-bootstrap-options-crmd-transition-delay"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       <primitive id="pDummy" class="ocf" provider="pacemaker" type="Dummy">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <operations>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           <op name="monitor" interval="10s" timeout="300s" on-fail="fence" id="pDummy-monitor-10s"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         </operations>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       </primitive>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       <group id="gStonith3">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <primitive id="f1" class="stonith" type="external/libvirt">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           <instance_attributes id="f1-instance_attributes">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++             <nvpair name="hostlist" value="vm3" id="f1-instance_attributes-hostlist"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++             <nvpair name="hypervisor_uri" value="qemu+ssh://bl460g1n6/system" id="f1-instance_attributes-hypervisor_uri"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           </instance_attributes>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           <operations>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++             <op name="start" interval="0s" timeout="60s" id="f1-start-0s"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++             <op name="monitor" interval="3600s" timeout="60s" id="f1-monitor-3600s"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++             <op name="stop" interval="0s" timeout="60s" id="f1-stop-0s"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           </operations>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         </primitive>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <primitive id="f2" class="stonith" type="external/ssh">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           <instance_attributes id="f2-instance_attributes">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++             <nvpair name="pcmk_reboot_retries" value="1" id="f2-instance_attributes-pcmk_reboot_retries"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++             <nvpair name="hostlist" value="vm3" id="f2-instance_attributes-hostlist"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           </instance_attributes>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           <operations>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++             <op name="start" interval="0s" timeout="60s" id="f2-start-0s"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++             <op name="monitor" interval="3600s" timeout="60s" id="f2-monitor-3600s"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++             <op name="stop" interval="0s" timeout="60s" id="f2-stop-0s"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           </operations>
Oct 15 15:16:21 vm1 crmd[14874]:     info: election_timer_cb: Election election-0 complete
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         </primitive>
Oct 15 15:16:21 vm1 crmd[14874]:     info: election_timeout_popped: Election failed: Declaring ourselves the winner
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       </group>
Oct 15 15:16:21 vm1 crmd[14874]:     info: do_log: FSA: Input I_ELECTION_DC from election_timeout_popped() received in state S_ELECTION
Oct 15 15:16:21 vm1 crmd[14874]:   notice: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_TIMER_POPPED origin=election_timeout_popped ]
Oct 15 15:16:21 vm1 crmd[14874]:     info: do_dc_takeover: Taking over DC status for this partition
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       <rsc_location id="l1" rsc="pDummy">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <rule score="300" id="l1-rule">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           <expression attribute="#uname" operation="eq" value="vm3" id="l1-expression"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         </rule>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       </rsc_location>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       <rsc_location id="l2" rsc="gStonith3">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <rule score="-INFINITY" id="l2-rule">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           <expression attribute="#uname" operation="eq" value="vm3" id="l2-expression"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         </rule>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <rule score="200" id="l2-rule-0">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++           <expression attribute="#uname" operation="eq" value="vm1" id="l2-expression-0"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         </rule>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       </rsc_location>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++     <fencing-topology>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       <fencing-level target="vm3" devices="f1" index="1" id="fencing"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       <fencing-level target="vm3" devices="f2" index="2" id="fencing-0"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++     </fencing-topology>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++     <rsc_defaults>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       <meta_attributes id="rsc-options">
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <nvpair name="resource-stickiness" value="INFINITY" id="rsc-options-resource-stickiness"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <nvpair name="migration-threshold" value="1" id="rsc-options-migration-threshold"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++       </meta_attributes>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++     </rsc_defaults>
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_replace operation for section 'all': OK (rc=0, origin=local/cibadmin/2, version=0.6.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/32, version=0.6.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section crm_config: OK (rc=0, origin=local/crmd/33, version=0.6.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=local/crmd/34, version=0.6.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/35, version=0.6.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_master operation for section 'all': OK (rc=0, origin=local/crmd/36, version=0.6.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/attrd/6, version=0.6.1)
Oct 15 15:16:21 vm1 attrd[14872]:     info: attrd_cib_callback: Update 6 for shutdown: OK (0)
Oct 15 15:16:21 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 6 for shutdown[vm1]=(null): OK (0)
Oct 15 15:16:21 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 6 for shutdown[vm2]=(null): OK (0)
Oct 15 15:16:21 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 6 for shutdown[vm3]=(null): OK (0)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/attrd/7, version=0.6.1)
Oct 15 15:16:21 vm1 attrd[14872]:     info: attrd_cib_callback: Update 7 for terminate: OK (0)
Oct 15 15:16:21 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 7 for terminate[vm1]=(null): OK (0)
Oct 15 15:16:21 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 7 for terminate[vm2]=(null): OK (0)
Oct 15 15:16:21 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 7 for terminate[vm3]=(null): OK (0)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/attrd/8, version=0.6.1)
Oct 15 15:16:21 vm1 attrd[14872]:     info: attrd_cib_callback: Update 8 for probe_complete: OK (0)
Oct 15 15:16:21 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 8 for probe_complete[vm1]=true: OK (0)
Oct 15 15:16:21 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 8 for probe_complete[vm2]=true: OK (0)
Oct 15 15:16:21 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 8 for probe_complete[vm3]=true: OK (0)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section cib: OK (rc=0, origin=local/crmd/37, version=0.6.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section //cib/configuration/crm_config//cluster_property_set//nvpair[@name='dc-version']: No such device or address (rc=-6, origin=local/crmd/38, version=0.6.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: crm_client_destroy: Destroying 0 events
Oct 15 15:16:21 vm1 cib[14869]:   notice: log_cib_diff: cib:diff: Local-only Change: 0.7.1
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: -- <cib admin_epoch="0" epoch="6" num_updates="1"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.11-0.284.6a5e863.git.el6-6a5e863"/>
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section crm_config: OK (rc=0, origin=local/crmd/39, version=0.7.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section //cib/configuration/crm_config//cluster_property_set//nvpair[@name='cluster-infrastructure']: No such device or address (rc=-6, origin=local/crmd/40, version=0.7.1)
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: initialize_join: Node vm3[3232261519] - join-3 phase 4 -> 0
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: initialize_join: Node vm1[3232261517] - join-3 phase 4 -> 0
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: initialize_join: Node vm2[3232261518] - join-3 phase 4 -> 0
Oct 15 15:16:21 vm1 crmd[14874]:     info: join_make_offer: join-3: Sending offer to vm3
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm3[3232261519] - join-3 phase 0 -> 1
Oct 15 15:16:21 vm1 crmd[14874]:     info: join_make_offer: join-3: Sending offer to vm1
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm1[3232261517] - join-3 phase 0 -> 1
Oct 15 15:16:21 vm1 crmd[14874]:     info: join_make_offer: join-3: Sending offer to vm2
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm2[3232261518] - join-3 phase 0 -> 1
Oct 15 15:16:21 vm1 crmd[14874]:     info: do_dc_join_offer_all: join-3: Waiting on 3 outstanding join acks
Oct 15 15:16:21 vm1 crmd[14874]:  warning: do_log: FSA: Input I_ELECTION_DC from do_election_check() received in state S_INTEGRATION
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: initialize_join: Node vm3[3232261519] - join-4 phase 1 -> 0
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: initialize_join: Node vm1[3232261517] - join-4 phase 1 -> 0
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: initialize_join: Node vm2[3232261518] - join-4 phase 1 -> 0
Oct 15 15:16:21 vm1 crmd[14874]:     info: join_make_offer: join-4: Sending offer to vm3
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm3[3232261519] - join-4 phase 0 -> 1
Oct 15 15:16:21 vm1 crmd[14874]:     info: join_make_offer: join-4: Sending offer to vm1
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm1[3232261517] - join-4 phase 0 -> 1
Oct 15 15:16:21 vm1 crmd[14874]:     info: join_make_offer: join-4: Sending offer to vm2
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: join_make_offer: Node vm2[3232261518] - join-4 phase 0 -> 1
Oct 15 15:16:21 vm1 crmd[14874]:     info: do_dc_join_offer_all: join-4: Waiting on 3 outstanding join acks
Oct 15 15:16:21 vm1 crmd[14874]:     info: update_dc: Set DC to vm1 (3.0.7)
Oct 15 15:16:21 vm1 cib[14869]:   notice: log_cib_diff: cib:diff: Local-only Change: 0.8.1
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: -- <cib admin_epoch="0" epoch="7" num_updates="1"/>
Oct 15 15:16:21 vm1 cib[14869]:   notice: cib:diff: ++         <nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/>
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section crm_config: OK (rc=0, origin=local/crmd/41, version=0.8.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section crm_config: OK (rc=0, origin=local/crmd/42, version=0.8.1)
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_filter_offer: Node vm3[3232261519] - join-4 phase 1 -> 2
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section crm_config: OK (rc=0, origin=local/crmd/43, version=0.8.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section crm_config: OK (rc=0, origin=local/crmd/44, version=0.8.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/45, version=0.8.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/46, version=0.8.1)
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_filter_offer: Node vm2[3232261518] - join-4 phase 1 -> 2
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_filter_offer: Node vm1[3232261517] - join-4 phase 1 -> 2
Oct 15 15:16:21 vm1 crmd[14874]:     info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Oct 15 15:16:21 vm1 crmd[14874]:     info: crmd_join_phase_log: join-4: vm3=integrated
Oct 15 15:16:21 vm1 crmd[14874]:     info: crmd_join_phase_log: join-4: vm1=integrated
Oct 15 15:16:21 vm1 crmd[14874]:     info: crmd_join_phase_log: join-4: vm2=integrated
Oct 15 15:16:21 vm1 crmd[14874]:     info: do_dc_join_finalize: join-4: Syncing our CIB to the rest of the cluster
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_sync operation for section 'all': OK (rc=0, origin=local/crmd/47, version=0.8.1)
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: finalize_join_for: Node vm3[3232261519] - join-4 phase 2 -> 3
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: finalize_join_for: Node vm1[3232261517] - join-4 phase 2 -> 3
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: finalize_join_for: Node vm2[3232261518] - join-4 phase 2 -> 3
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=local/crmd/48, version=0.8.1)
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_ack: Node vm3[3232261519] - join-4 phase 3 -> 4
Oct 15 15:16:21 vm1 crmd[14874]:     info: do_dc_join_ack: join-4: Updating node state to member for vm3
Oct 15 15:16:21 vm1 crmd[14874]:     info: erase_status_tag: Deleting xpath: //node_state[@uname='vm3']/lrm
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_ack: Node vm1[3232261517] - join-4 phase 3 -> 4
Oct 15 15:16:21 vm1 crmd[14874]:     info: do_dc_join_ack: join-4: Updating node state to member for vm1
Oct 15 15:16:21 vm1 crmd[14874]:     info: erase_status_tag: Deleting xpath: //node_state[@uname='vm1']/lrm
Oct 15 15:16:21 vm1 crmd[14874]:     info: crm_update_peer_join: do_dc_join_ack: Node vm2[3232261518] - join-4 phase 3 -> 4
Oct 15 15:16:21 vm1 crmd[14874]:     info: do_dc_join_ack: join-4: Updating node state to member for vm2
Oct 15 15:16:21 vm1 crmd[14874]:     info: erase_status_tag: Deleting xpath: //node_state[@uname='vm2']/lrm
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=local/crmd/49, version=0.8.1)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=local/crmd/50, version=0.8.1)
Oct 15 15:16:21 vm1 cib[14925]:     info: write_cib_contents: Archived previous version as /var/lib/pacemaker/cib/cib-2.raw
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='vm3']/lrm: OK (rc=0, origin=local/crmd/51, version=0.8.2)
Oct 15 15:16:21 vm1 cib[14925]:     info: write_cib_contents: Wrote version 0.8.0 of the CIB to disk (digest: 560938b85dddf6a8da1def7aa23e6520)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/52, version=0.8.3)
Oct 15 15:16:21 vm1 cib[14925]:     info: retrieveCib: Reading cluster configuration from: /var/lib/pacemaker/cib/cib.3CBBuO (digest: /var/lib/pacemaker/cib/cib.2ucdvm)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='vm1']/lrm: OK (rc=0, origin=local/crmd/53, version=0.8.4)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/54, version=0.8.5)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_delete operation for section //node_state[@uname='vm2']/lrm: OK (rc=0, origin=local/crmd/55, version=0.8.6)
Oct 15 15:16:21 vm1 crmd[14874]:     info: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Oct 15 15:16:21 vm1 crmd[14874]:     info: abort_transition_graph: do_te_invoke:151 - Triggered transition abort (complete=1) : Peer Cancelled
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/56, version=0.8.7)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section nodes: OK (rc=0, origin=local/crmd/57, version=0.8.7)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/58, version=0.8.7)
Oct 15 15:16:21 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section cib: OK (rc=0, origin=local/crmd/59, version=0.8.7)
Oct 15 15:16:22 vm1 cib[14869]:     info: crm_client_new: Connecting 0x144af10 for uid=0 gid=0 pid=14924 id=d1e89bc1-2943-4921-afc4-d159e63cd0a8
Oct 15 15:16:22 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/cibadmin/2, version=0.8.7)
Oct 15 15:16:22 vm1 cib[14869]:     info: crm_client_destroy: Destroying 0 events
Oct 15 15:16:22 vm1 stonith-ng[14870]:   notice: stonith_device_register: Added 'f1' to the device list (1 active devices)
Oct 15 15:16:22 vm1 stonith-ng[14870]:     info: cib_device_update: Device f2 is allowed on vm1: score=0
Oct 15 15:16:22 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action metadata for agent fence_legacy (target=(null))
Oct 15 15:16:23 vm1 stonith-ng[14870]:   notice: stonith_device_register: Added 'f2' to the device list (2 active devices)
Oct 15 15:16:23 vm1 crmd[14874]:     info: crm_timer_popped: New Transition Timer (I_PE_CALC) just popped (2000ms)
Oct 15 15:16:23 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/60, version=0.8.7)
Oct 15 15:16:23 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm2 is active
Oct 15 15:16:23 vm1 pengine[14873]:     info: determine_online_status: Node vm2 is online
Oct 15 15:16:23 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm3 is active
Oct 15 15:16:23 vm1 pengine[14873]:     info: determine_online_status: Node vm3 is online
Oct 15 15:16:23 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm1 is active
Oct 15 15:16:23 vm1 pengine[14873]:     info: determine_online_status: Node vm1 is online
Oct 15 15:16:23 vm1 pengine[14873]:     info: native_print: pDummy	(ocf::pacemaker:Dummy):	Stopped 
Oct 15 15:16:23 vm1 pengine[14873]:     info: group_print:  Resource Group: gStonith3
Oct 15 15:16:23 vm1 pengine[14873]:     info: native_print:      f1	(stonith:external/libvirt):	Stopped 
Oct 15 15:16:23 vm1 pengine[14873]:     info: native_print:      f2	(stonith:external/ssh):	Stopped 
Oct 15 15:16:23 vm1 pengine[14873]:     info: RecurringOp:  Start recurring monitor (10s) for pDummy on vm3
Oct 15 15:16:23 vm1 pengine[14873]:     info: RecurringOp:  Start recurring monitor (3600s) for f1 on vm1
Oct 15 15:16:23 vm1 pengine[14873]:     info: RecurringOp:  Start recurring monitor (3600s) for f2 on vm1
Oct 15 15:16:23 vm1 pengine[14873]:   notice: LogActions: Start   pDummy	(vm3)
Oct 15 15:16:23 vm1 pengine[14873]:   notice: LogActions: Start   f1	(vm1)
Oct 15 15:16:23 vm1 pengine[14873]:   notice: LogActions: Start   f2	(vm1)
Oct 15 15:16:24 vm1 crmd[14874]:     info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Oct 15 15:16:24 vm1 crmd[14874]:     info: do_te_invoke: Processing graph 1 (ref=pe_calc-dc-1381817783-33) derived from /var/lib/pacemaker/pengine/pe-input-1.bz2
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 12: monitor pDummy_monitor_0 on vm3
Oct 15 15:16:24 vm1 pengine[14873]:   notice: process_pe_message: Calculated Transition 1: /var/lib/pacemaker/pengine/pe-input-1.bz2
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 8: monitor pDummy_monitor_0 on vm2
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 4: monitor pDummy_monitor_0 on vm1 (local)
Oct 15 15:16:24 vm1 lrmd[14871]:     info: process_lrmd_get_rsc_info: Resource 'pDummy' not found (0 active resources)
Oct 15 15:16:24 vm1 lrmd[14871]:     info: process_lrmd_rsc_register: Added 'pDummy' to the rsc list (1 active resources)
Oct 15 15:16:24 vm1 crmd[14874]:     info: do_lrm_rsc_op: Performing key=4:1:7:cffe5b98-3c92-4ed3-8992-426ef00df4ed op=pDummy_monitor_0
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 13: monitor f1_monitor_0 on vm3
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 9: monitor f1_monitor_0 on vm2
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 5: monitor f1_monitor_0 on vm1 (local)
Oct 15 15:16:24 vm1 lrmd[14871]:     info: process_lrmd_get_rsc_info: Resource 'f1' not found (1 active resources)
Oct 15 15:16:24 vm1 lrmd[14871]:     info: process_lrmd_rsc_register: Added 'f1' to the rsc list (2 active resources)
Oct 15 15:16:24 vm1 crmd[14874]:     info: do_lrm_rsc_op: Performing key=5:1:7:cffe5b98-3c92-4ed3-8992-426ef00df4ed op=f1_monitor_0
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 14: monitor f2_monitor_0 on vm3
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 10: monitor f2_monitor_0 on vm2
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 6: monitor f2_monitor_0 on vm1 (local)
Oct 15 15:16:24 vm1 stonith-ng[14870]:     info: crm_client_new: Connecting 0xd8c2a0 for uid=0 gid=0 pid=14871 id=8326641a-5ab4-4fc9-971f-95fa8e280da8
Oct 15 15:16:24 vm1 stonith-ng[14870]:     info: stonith_command: Processed register from lrmd.14871: OK (0)
Oct 15 15:16:24 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_notify from lrmd.14871: OK (0)
Oct 15 15:16:24 vm1 lrmd[14871]:     info: process_lrmd_get_rsc_info: Resource 'f2' not found (2 active resources)
Oct 15 15:16:24 vm1 lrmd[14871]:     info: process_lrmd_rsc_register: Added 'f2' to the rsc list (3 active resources)
Oct 15 15:16:24 vm1 crmd[14874]:     info: do_lrm_rsc_op: Performing key=6:1:7:cffe5b98-3c92-4ed3-8992-426ef00df4ed op=f2_monitor_0
Oct 15 15:16:24 vm1 Dummy(pDummy)[14927]: DEBUG: pDummy monitor : 7
Oct 15 15:16:24 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vm2/crmd/13, version=0.8.8)
Oct 15 15:16:24 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vm3/crmd/13, version=0.8.9)
Oct 15 15:16:24 vm1 crmd[14874]:     info: process_lrm_event: LRM operation f1_monitor_0 (call=9, rc=7, cib-update=61, confirmed=true) not running
Oct 15 15:16:24 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/61, version=0.8.10)
Oct 15 15:16:24 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vm3/crmd/14, version=0.8.11)
Oct 15 15:16:24 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vm2/crmd/14, version=0.8.12)
Oct 15 15:16:24 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vm3/crmd/15, version=0.8.13)
Oct 15 15:16:24 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vm2/crmd/15, version=0.8.14)
Oct 15 15:16:24 vm1 crmd[14874]:     info: process_lrm_event: LRM operation f2_monitor_0 (call=13, rc=7, cib-update=62, confirmed=true) not running
Oct 15 15:16:24 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/62, version=0.8.15)
Oct 15 15:16:24 vm1 crmd[14874]:     info: services_os_action_execute: Managed Dummy_meta-data_0 process 14961 exited with rc=0
Oct 15 15:16:24 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/63, version=0.8.16)
Oct 15 15:16:24 vm1 crmd[14874]:   notice: process_lrm_event: LRM operation pDummy_monitor_0 (call=5, rc=7, cib-update=63, confirmed=true) not running
Oct 15 15:16:24 vm1 crmd[14874]:     info: match_graph_event: Action f1_monitor_0 (9) confirmed on vm2 (rc=0)
Oct 15 15:16:24 vm1 crmd[14874]:     info: match_graph_event: Action f1_monitor_0 (13) confirmed on vm3 (rc=0)
Oct 15 15:16:24 vm1 crmd[14874]:     info: match_graph_event: Action f1_monitor_0 (5) confirmed on vm1 (rc=0)
Oct 15 15:16:24 vm1 crmd[14874]:     info: match_graph_event: Action f2_monitor_0 (14) confirmed on vm3 (rc=0)
Oct 15 15:16:24 vm1 crmd[14874]:     info: match_graph_event: Action f2_monitor_0 (10) confirmed on vm2 (rc=0)
Oct 15 15:16:24 vm1 crmd[14874]:     info: match_graph_event: Action pDummy_monitor_0 (12) confirmed on vm3 (rc=0)
Oct 15 15:16:24 vm1 crmd[14874]:     info: match_graph_event: Action pDummy_monitor_0 (8) confirmed on vm2 (rc=0)
Oct 15 15:16:24 vm1 crmd[14874]:     info: match_graph_event: Action f2_monitor_0 (6) confirmed on vm1 (rc=0)
Oct 15 15:16:24 vm1 crmd[14874]:     info: match_graph_event: Action pDummy_monitor_0 (4) confirmed on vm1 (rc=0)
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 11: probe_complete probe_complete on vm3 - no waiting
Oct 15 15:16:24 vm1 crmd[14874]:     info: te_rsc_command: Action 11 confirmed - no wait
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 7: probe_complete probe_complete on vm2 - no waiting
Oct 15 15:16:24 vm1 crmd[14874]:     info: te_rsc_command: Action 7 confirmed - no wait
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 3: probe_complete probe_complete on vm1 (local) - no waiting
Oct 15 15:16:24 vm1 attrd[14872]:     info: attrd_client_message: Broadcasting probe_complete[vm1] = true (writer)
Oct 15 15:16:24 vm1 crmd[14874]:     info: te_rsc_command: Action 3 confirmed - no wait
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 15: start pDummy_start_0 on vm3
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 17: start f1_start_0 on vm1 (local)
Oct 15 15:16:24 vm1 crmd[14874]:     info: do_lrm_rsc_op: Performing key=17:1:0:cffe5b98-3c92-4ed3-8992-426ef00df4ed op=f1_start_0
Oct 15 15:16:24 vm1 lrmd[14871]:     info: log_execute: executing - rsc:f1 action:start call_id:14
Oct 15 15:16:24 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action metadata for agent fence_legacy (target=(null))
Oct 15 15:16:24 vm1 crmd[14874]:     info: match_graph_event: Action pDummy_start_0 (15) confirmed on vm3 (rc=0)
Oct 15 15:16:24 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 16: monitor pDummy_monitor_10000 on vm3
Oct 15 15:16:24 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vm3/crmd/16, version=0.8.17)
Oct 15 15:16:24 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vm3/crmd/17, version=0.8.18)
Oct 15 15:16:24 vm1 crmd[14874]:     info: match_graph_event: Action pDummy_monitor_10000 (16) confirmed on vm3 (rc=0)
Oct 15 15:16:25 vm1 stonith-ng[14870]:   notice: stonith_device_register: Device 'f1' already existed in device list (2 active devices)
Oct 15 15:16:25 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_device_register from lrmd.14871: OK (0)
Oct 15 15:16:25 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_execute from lrmd.14871: Operation now in progress (-115)
Oct 15 15:16:25 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action monitor for agent fence_legacy (target=(null))
Oct 15 15:16:26 vm1 stonith-ng[14870]:     info: log_operation: f1:14966 [ Performing: stonith -t external/libvirt -S ]
Oct 15 15:16:26 vm1 stonith-ng[14870]:     info: log_operation: f1:14966 [ success:  0 ]
Oct 15 15:16:26 vm1 lrmd[14871]:     info: log_finished: finished - rsc:f1 action:start call_id:14  exit-code:0 exec-time:2339ms queue-time:0ms
Oct 15 15:16:26 vm1 crmd[14874]:   notice: process_lrm_event: LRM operation f1_start_0 (call=14, rc=0, cib-update=64, confirmed=true) ok
Oct 15 15:16:26 vm1 crmd[14874]:     info: match_graph_event: Action f1_start_0 (17) confirmed on vm1 (rc=0)
Oct 15 15:16:26 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 18: monitor f1_monitor_3600000 on vm1 (local)
Oct 15 15:16:26 vm1 crmd[14874]:     info: do_lrm_rsc_op: Performing key=18:1:0:cffe5b98-3c92-4ed3-8992-426ef00df4ed op=f1_monitor_3600000
Oct 15 15:16:26 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/64, version=0.8.19)
Oct 15 15:16:26 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_execute from lrmd.14871: Operation now in progress (-115)
Oct 15 15:16:26 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 19: start f2_start_0 on vm1 (local)
Oct 15 15:16:26 vm1 crmd[14874]:     info: do_lrm_rsc_op: Performing key=19:1:0:cffe5b98-3c92-4ed3-8992-426ef00df4ed op=f2_start_0
Oct 15 15:16:26 vm1 lrmd[14871]:     info: log_execute: executing - rsc:f2 action:start call_id:16
Oct 15 15:16:26 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action monitor for agent fence_legacy (target=(null))
Oct 15 15:16:26 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action metadata for agent fence_legacy (target=(null))
Oct 15 15:16:26 vm1 stonith-ng[14870]:   notice: stonith_device_register: Device 'f2' already existed in device list (2 active devices)
Oct 15 15:16:26 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_device_register from lrmd.14871: OK (0)
Oct 15 15:16:26 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_execute from lrmd.14871: Operation now in progress (-115)
Oct 15 15:16:26 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action monitor for agent fence_legacy (target=(null))
Oct 15 15:16:27 vm1 stonith-ng[14870]:     info: log_operation: f2:14985 [ Performing: stonith -t external/ssh -S ]
Oct 15 15:16:27 vm1 stonith-ng[14870]:     info: log_operation: f2:14985 [ success:  0 ]
Oct 15 15:16:27 vm1 lrmd[14871]:     info: log_finished: finished - rsc:f2 action:start call_id:16  exit-code:0 exec-time:1215ms queue-time:1ms
Oct 15 15:16:27 vm1 crmd[14874]:   notice: process_lrm_event: LRM operation f2_start_0 (call=16, rc=0, cib-update=65, confirmed=true) ok
Oct 15 15:16:27 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/65, version=0.8.20)
Oct 15 15:16:27 vm1 crmd[14874]:     info: match_graph_event: Action f2_start_0 (19) confirmed on vm1 (rc=0)
Oct 15 15:16:27 vm1 crmd[14874]:   notice: te_rsc_command: Initiating action 20: monitor f2_monitor_3600000 on vm1 (local)
Oct 15 15:16:27 vm1 crmd[14874]:     info: do_lrm_rsc_op: Performing key=20:1:0:cffe5b98-3c92-4ed3-8992-426ef00df4ed op=f2_monitor_3600000
Oct 15 15:16:27 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_execute from lrmd.14871: Operation now in progress (-115)
Oct 15 15:16:27 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action monitor for agent fence_legacy (target=(null))
Oct 15 15:16:28 vm1 stonith-ng[14870]:     info: log_operation: f1:14982 [ Performing: stonith -t external/libvirt -S ]
Oct 15 15:16:28 vm1 stonith-ng[14870]:     info: log_operation: f1:14982 [ success:  0 ]
Oct 15 15:16:28 vm1 crmd[14874]:   notice: process_lrm_event: LRM operation f1_monitor_3600000 (call=15, rc=0, cib-update=66, confirmed=false) ok
Oct 15 15:16:28 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/66, version=0.8.21)
Oct 15 15:16:28 vm1 crmd[14874]:     info: match_graph_event: Action f1_monitor_3600000 (18) confirmed on vm1 (rc=0)
Oct 15 15:16:29 vm1 stonith-ng[14870]:     info: log_operation: f2:15011 [ Performing: stonith -t external/ssh -S ]
Oct 15 15:16:29 vm1 stonith-ng[14870]:     info: log_operation: f2:15011 [ success:  0 ]
Oct 15 15:16:29 vm1 crmd[14874]:   notice: process_lrm_event: LRM operation f2_monitor_3600000 (call=17, rc=0, cib-update=67, confirmed=false) ok
Oct 15 15:16:29 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/crmd/67, version=0.8.22)
Oct 15 15:16:29 vm1 crmd[14874]:     info: match_graph_event: Action f2_monitor_3600000 (20) confirmed on vm1 (rc=0)
Oct 15 15:16:29 vm1 crmd[14874]:   notice: run_graph: Transition 1 (Complete=21, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1.bz2): Complete
Oct 15 15:16:29 vm1 crmd[14874]:     info: do_log: FSA: Input I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE
Oct 15 15:16:29 vm1 crmd[14874]:   notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Oct 15 15:16:37 vm1 cib[14869]:     info: crm_client_new: Connecting 0x143a6c0 for uid=0 gid=0 pid=15029 id=384ff994-a623-4790-8ad1-a66967c08079
Oct 15 15:16:37 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crm_mon/2, version=0.8.22)
Oct 15 15:16:37 vm1 cib[14869]:     info: crm_client_destroy: Destroying 0 events
Oct 15 15:17:04 vm1 crmd[14874]:     info: abort_transition_graph: process_graph_event:583 - Triggered transition abort (complete=1, node=vm3, tag=lrm_rsc_op, id=pDummy_last_failure_0, magic=0:7;16:1:0:cffe5b98-3c92-4ed3-8992-426ef00df4ed, cib=0.8.23) : Inactive graph
Oct 15 15:17:04 vm1 crmd[14874]:  warning: update_failcount: Updating failcount for pDummy on vm3 after failed monitor: rc=7 (update=value++, time=1381817824)
Oct 15 15:17:04 vm1 crmd[14874]:     info: process_graph_event: Detected action (1.16) pDummy_monitor_10000.15=not running: failed
Oct 15 15:17:04 vm1 attrd[14872]:     info: attrd_client_message: Expanded fail-count-pDummy=value++ to 1
Oct 15 15:17:04 vm1 attrd[14872]:     info: attrd_client_message: Broadcasting fail-count-pDummy[vm3] = 1 (writer)
Oct 15 15:17:04 vm1 attrd[14872]:     info: attrd_client_message: Broadcasting last-failure-pDummy[vm3] = 1381817824 (writer)
Oct 15 15:17:04 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=vm3/crmd/18, version=0.8.23)
Oct 15 15:17:04 vm1 attrd[14872]:   notice: write_attribute: Sent update 9 with 1 changes for fail-count-pDummy, id=<n/a>, set=(null)
Oct 15 15:17:04 vm1 attrd[14872]:   notice: write_attribute: Sent update 10 with 1 changes for last-failure-pDummy, id=<n/a>, set=(null)
Oct 15 15:17:04 vm1 crmd[14874]:     info: abort_transition_graph: te_update_diff:172 - Triggered transition abort (complete=1, node=vm3, tag=nvpair, id=status-3232261519-fail-count-pDummy, name=fail-count-pDummy, value=1, magic=NA, cib=0.8.24) : Transient attribute: update
Oct 15 15:17:04 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/attrd/9, version=0.8.24)
Oct 15 15:17:04 vm1 crmd[14874]:     info: abort_transition_graph: te_update_diff:172 - Triggered transition abort (complete=1, node=vm3, tag=nvpair, id=status-3232261519-last-failure-pDummy, name=last-failure-pDummy, value=1381817824, magic=NA, cib=0.8.25) : Transient attribute: update
Oct 15 15:17:04 vm1 cib[14869]:     info: cib_process_request: Completed cib_modify operation for section status: OK (rc=0, origin=local/attrd/10, version=0.8.25)
Oct 15 15:17:04 vm1 attrd[14872]:     info: attrd_cib_callback: Update 9 for fail-count-pDummy: OK (0)
Oct 15 15:17:04 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 9 for fail-count-pDummy[vm3]=1: OK (0)
Oct 15 15:17:04 vm1 attrd[14872]:     info: attrd_cib_callback: Update 10 for last-failure-pDummy: OK (0)
Oct 15 15:17:04 vm1 attrd[14872]:   notice: attrd_cib_callback: Update 10 for last-failure-pDummy[vm3]=1381817824: OK (0)
Oct 15 15:17:06 vm1 crmd[14874]:     info: crm_timer_popped: New Transition Timer (I_PE_CALC) just popped (2000ms)
Oct 15 15:17:06 vm1 crmd[14874]:   notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]
Oct 15 15:17:06 vm1 crmd[14874]:     info: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED
Oct 15 15:17:06 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/68, version=0.8.25)
Oct 15 15:17:06 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm2 is active
Oct 15 15:17:06 vm1 pengine[14873]:     info: determine_online_status: Node vm2 is online
Oct 15 15:17:06 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm3 is active
Oct 15 15:17:06 vm1 pengine[14873]:     info: determine_online_status: Node vm3 is online
Oct 15 15:17:06 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm1 is active
Oct 15 15:17:06 vm1 pengine[14873]:     info: determine_online_status: Node vm1 is online
Oct 15 15:17:06 vm1 pengine[14873]:  warning: unpack_rsc_op_failure: Processing failed op monitor for pDummy on vm3: not running (7)
Oct 15 15:17:06 vm1 pengine[14873]:  warning: pe_fence_node: Node vm3 will be fenced because of resource failure(s)
Oct 15 15:17:06 vm1 pengine[14873]:     info: native_print: pDummy	(ocf::pacemaker:Dummy):	FAILED vm3 
Oct 15 15:17:06 vm1 pengine[14873]:     info: group_print:  Resource Group: gStonith3
Oct 15 15:17:06 vm1 pengine[14873]:     info: native_print:      f1	(stonith:external/libvirt):	Started vm1 
Oct 15 15:17:06 vm1 pengine[14873]:     info: native_print:      f2	(stonith:external/ssh):	Started vm1 
Oct 15 15:17:06 vm1 pengine[14873]:     info: get_failcount_full: pDummy has failed 1 times on vm3
Oct 15 15:17:06 vm1 pengine[14873]:  warning: common_apply_stickiness: Forcing pDummy away from vm3 after 1 failures (max=1)
Oct 15 15:17:06 vm1 pengine[14873]:     info: RecurringOp:  Start recurring monitor (10s) for pDummy on vm1
Oct 15 15:17:06 vm1 pengine[14873]:  warning: stage6: Scheduling Node vm3 for STONITH
Oct 15 15:17:06 vm1 pengine[14873]:   notice: native_stop_constraints: Stop of failed resource pDummy is implicit after vm3 is fenced
Oct 15 15:17:06 vm1 pengine[14873]:   notice: LogActions: Recover pDummy	(Started vm3 -> vm1)
Oct 15 15:17:06 vm1 pengine[14873]:     info: LogActions: Leave   f1	(Started vm1)
Oct 15 15:17:06 vm1 pengine[14873]:     info: LogActions: Leave   f2	(Started vm1)
Oct 15 15:17:06 vm1 pengine[14873]:  warning: process_pe_message: Calculated Transition 2: /var/lib/pacemaker/pengine/pe-warn-0.bz2
Oct 15 15:17:06 vm1 crmd[14874]:     info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Oct 15 15:17:06 vm1 crmd[14874]:     info: do_te_invoke: Processing graph 2 (ref=pe_calc-dc-1381817826-52) derived from /var/lib/pacemaker/pengine/pe-warn-0.bz2
Oct 15 15:17:06 vm1 crmd[14874]:   notice: te_fence_node: Executing reboot fencing operation (20) on vm3 (timeout=60000)
Oct 15 15:17:06 vm1 stonith-ng[14870]:   notice: handle_request: Client crmd.14874.4eb0ff33 wants to fence (reboot) 'vm3' with device '(any)'
Oct 15 15:17:06 vm1 stonith-ng[14870]:   notice: initiate_remote_stonith_op: Initiating remote operation reboot for vm3: c9b3e4f1-269f-48a8-ba27-c7573dead8e2 (0)
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_fence from crmd.14874: Operation now in progress (-115)
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query from vm1: OK (0)
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action list for agent fence_legacy (target=(null))
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action list for agent fence_legacy (target=(null))
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: process_remote_stonith_query: Ignoring reply from vm3, hosts are not permitted to commit suicide
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query reply from vm3: OK (0)
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: process_remote_stonith_query: Query result 2 of 3 from vm2 (2 devices)
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: call_remote_stonith: Total remote op timeout set to 120 for fencing of node vm3 for crmd.14874.c9b3e4f1
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: call_remote_stonith: Requesting that vm2 perform op reboot vm3 with f1 for crmd.14874 (72s)
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query reply from vm2: OK (0)
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: dynamic_list_search_cb: Refreshing port list for f2
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: dynamic_list_search_cb: Refreshing port list for f1
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: process_remote_stonith_query: Query result 3 of 3 from vm1 (2 devices)
Oct 15 15:17:06 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query reply from vm1: OK (0)
Oct 15 15:17:16 vm1 stonith-ng[14870]:   notice: process_remote_stonith_exec: Call to f1 for vm3 on behalf of crmd.14874@vm1: Generic Pacemaker error (-201)
Oct 15 15:17:16 vm1 stonith-ng[14870]:     info: call_remote_stonith: Requesting that vm1 perform op reboot vm3 with f2 for crmd.14874 (72s)
Oct 15 15:17:16 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_fence reply from vm2: OK (0)
Oct 15 15:17:16 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_fence from vm1: Operation now in progress (-115)
Oct 15 15:17:16 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action reboot for agent fence_legacy (target=vm3)
Oct 15 15:17:35 vm1 stonith: [15077]: CRIT: external_reset_req: 'ssh reset' for host vm3 failed with rc 1
Oct 15 15:17:35 vm1 stonith-ng[14870]:     info: update_remaining_timeout: Attempted to execute agent fence_legacy (reboot) the maximum number of times (1) allowed
Oct 15 15:17:35 vm1 stonith-ng[14870]:    error: log_operation: Operation 'reboot' [15076] (call 2 from crmd.14874) for host 'vm3' with device 'f2' returned: -201 (Generic Pacemaker error)
Oct 15 15:17:35 vm1 stonith-ng[14870]:  warning: log_operation: f2:15076 [ Performing: stonith -t external/ssh -T reset vm3 ]
Oct 15 15:17:35 vm1 stonith-ng[14870]:  warning: log_operation: f2:15076 [ failed: vm3 5 ]
Oct 15 15:17:35 vm1 stonith-ng[14870]:   notice: process_remote_stonith_exec: Call to f2 for vm3 on behalf of crmd.14874@vm1: Generic Pacemaker error (-201)
Oct 15 15:17:35 vm1 stonith-ng[14870]:   notice: stonith_topology_next: All fencing options to fence vm3 for crmd.14874@vm1.c9b3e4f1 failed
Oct 15 15:17:35 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_fence reply from vm1: OK (0)
Oct 15 15:17:35 vm1 stonith-ng[14870]:    error: remote_op_done: Operation reboot of vm3 by vm1 for crmd.14874@vm1.c9b3e4f1: Generic Pacemaker error
Oct 15 15:17:35 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_notify reply from vm1: OK (0)
Oct 15 15:17:35 vm1 crmd[14874]:   notice: tengine_stonith_callback: Stonith operation 2/20:2:0:cffe5b98-3c92-4ed3-8992-426ef00df4ed: Generic Pacemaker error (-201)
Oct 15 15:17:35 vm1 crmd[14874]:   notice: tengine_stonith_callback: Stonith operation 2 for vm3 failed (Generic Pacemaker error): aborting transition.
Oct 15 15:17:35 vm1 crmd[14874]:     info: abort_transition_graph: tengine_stonith_callback:463 - Triggered transition abort (complete=0) : Stonith failed
Oct 15 15:17:35 vm1 crmd[14874]:   notice: tengine_stonith_notify: Peer vm3 was not terminated (reboot) by vm1 for vm1: Generic Pacemaker error (ref=c9b3e4f1-269f-48a8-ba27-c7573dead8e2) by client crmd.14874
Oct 15 15:17:35 vm1 crmd[14874]:   notice: run_graph: Transition 2 (Complete=1, Pending=0, Fired=0, Skipped=5, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-warn-0.bz2): Stopped
Oct 15 15:17:37 vm1 crmd[14874]:     info: crm_timer_popped: New Transition Timer (I_PE_CALC) just popped (2000ms)
Oct 15 15:17:37 vm1 crmd[14874]:     info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]
Oct 15 15:17:37 vm1 crmd[14874]:     info: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED
Oct 15 15:17:37 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/69, version=0.8.25)
Oct 15 15:17:37 vm1 pengine[14873]:     info: process_pe_message: Input has not changed since last time, not saving to disk
Oct 15 15:17:37 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm2 is active
Oct 15 15:17:37 vm1 pengine[14873]:     info: determine_online_status: Node vm2 is online
Oct 15 15:17:37 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm3 is active
Oct 15 15:17:37 vm1 pengine[14873]:     info: determine_online_status: Node vm3 is online
Oct 15 15:17:37 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm1 is active
Oct 15 15:17:37 vm1 pengine[14873]:     info: determine_online_status: Node vm1 is online
Oct 15 15:17:37 vm1 pengine[14873]:  warning: unpack_rsc_op_failure: Processing failed op monitor for pDummy on vm3: not running (7)
Oct 15 15:17:37 vm1 pengine[14873]:  warning: pe_fence_node: Node vm3 will be fenced because of resource failure(s)
Oct 15 15:17:37 vm1 pengine[14873]:     info: native_print: pDummy	(ocf::pacemaker:Dummy):	FAILED vm3 
Oct 15 15:17:37 vm1 pengine[14873]:     info: group_print:  Resource Group: gStonith3
Oct 15 15:17:37 vm1 pengine[14873]:     info: native_print:      f1	(stonith:external/libvirt):	Started vm1 
Oct 15 15:17:37 vm1 pengine[14873]:     info: native_print:      f2	(stonith:external/ssh):	Started vm1 
Oct 15 15:17:37 vm1 pengine[14873]:     info: get_failcount_full: pDummy has failed 1 times on vm3
Oct 15 15:17:37 vm1 pengine[14873]:  warning: common_apply_stickiness: Forcing pDummy away from vm3 after 1 failures (max=1)
Oct 15 15:17:37 vm1 pengine[14873]:     info: RecurringOp:  Start recurring monitor (10s) for pDummy on vm1
Oct 15 15:17:37 vm1 pengine[14873]:  warning: stage6: Scheduling Node vm3 for STONITH
Oct 15 15:17:37 vm1 pengine[14873]:   notice: native_stop_constraints: Stop of failed resource pDummy is implicit after vm3 is fenced
Oct 15 15:17:37 vm1 pengine[14873]:   notice: LogActions: Recover pDummy	(Started vm3 -> vm1)
Oct 15 15:17:37 vm1 pengine[14873]:     info: LogActions: Leave   f1	(Started vm1)
Oct 15 15:17:37 vm1 pengine[14873]:     info: LogActions: Leave   f2	(Started vm1)
Oct 15 15:17:37 vm1 crmd[14874]:     info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Oct 15 15:17:37 vm1 crmd[14874]:     info: do_te_invoke: Processing graph 3 (ref=pe_calc-dc-1381817857-53) derived from /var/lib/pacemaker/pengine/pe-warn-0.bz2
Oct 15 15:17:37 vm1 crmd[14874]:   notice: te_fence_node: Executing reboot fencing operation (20) on vm3 (timeout=60000)
Oct 15 15:17:37 vm1 pengine[14873]:  warning: process_pe_message: Calculated Transition 3: /var/lib/pacemaker/pengine/pe-warn-0.bz2
Oct 15 15:17:37 vm1 stonith-ng[14870]:   notice: handle_request: Client crmd.14874.4eb0ff33 wants to fence (reboot) 'vm3' with device '(any)'
Oct 15 15:17:37 vm1 stonith-ng[14870]:   notice: initiate_remote_stonith_op: Initiating remote operation reboot for vm3: d5bd243d-da15-4098-80ab-c9f1bce3827f (0)
Oct 15 15:17:37 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_fence from crmd.14874: Operation now in progress (-115)
Oct 15 15:17:37 vm1 stonith-ng[14870]:   notice: can_fence_host_with_device: f1 can fence vm3: dynamic-list
Oct 15 15:17:37 vm1 stonith-ng[14870]:   notice: can_fence_host_with_device: f2 can fence vm3: dynamic-list
Oct 15 15:17:37 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query from vm1: OK (0)
Oct 15 15:17:37 vm1 stonith-ng[14870]:     info: process_remote_stonith_query: Query result 1 of 3 from vm2 (2 devices)
Oct 15 15:17:37 vm1 stonith-ng[14870]:     info: call_remote_stonith: Total remote op timeout set to 120 for fencing of node vm3 for crmd.14874.d5bd243d
Oct 15 15:17:37 vm1 stonith-ng[14870]:     info: call_remote_stonith: Requesting that vm2 perform op reboot vm3 with f1 for crmd.14874 (72s)
Oct 15 15:17:37 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query reply from vm2: OK (0)
Oct 15 15:17:37 vm1 stonith-ng[14870]:     info: process_remote_stonith_query: Ignoring reply from vm3, hosts are not permitted to commit suicide
Oct 15 15:17:37 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query reply from vm3: OK (0)
Oct 15 15:17:37 vm1 stonith-ng[14870]:     info: process_remote_stonith_query: Query result 3 of 3 from vm1 (2 devices)
Oct 15 15:17:37 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query reply from vm1: OK (0)
Oct 15 15:17:46 vm1 stonith-ng[14870]:   notice: process_remote_stonith_exec: Call to f1 for vm3 on behalf of crmd.14874@vm1: Generic Pacemaker error (-201)
Oct 15 15:17:46 vm1 stonith-ng[14870]:     info: call_remote_stonith: Requesting that vm1 perform op reboot vm3 with f2 for crmd.14874 (72s)
Oct 15 15:17:46 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_fence reply from vm2: OK (0)
Oct 15 15:17:46 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_fence from vm1: Operation now in progress (-115)
Oct 15 15:17:46 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action reboot for agent fence_legacy (target=vm3)
Oct 15 15:18:06 vm1 stonith: [15465]: CRIT: external_reset_req: 'ssh reset' for host vm3 failed with rc 1
Oct 15 15:18:06 vm1 stonith-ng[14870]:     info: update_remaining_timeout: Attempted to execute agent fence_legacy (reboot) the maximum number of times (1) allowed
Oct 15 15:18:06 vm1 stonith-ng[14870]:    error: log_operation: Operation 'reboot' [15464] (call 3 from crmd.14874) for host 'vm3' with device 'f2' returned: -201 (Generic Pacemaker error)
Oct 15 15:18:06 vm1 stonith-ng[14870]:  warning: log_operation: f2:15464 [ Performing: stonith -t external/ssh -T reset vm3 ]
Oct 15 15:18:06 vm1 stonith-ng[14870]:  warning: log_operation: f2:15464 [ failed: vm3 5 ]
Oct 15 15:18:06 vm1 stonith-ng[14870]:   notice: process_remote_stonith_exec: Call to f2 for vm3 on behalf of crmd.14874@vm1: Generic Pacemaker error (-201)
Oct 15 15:18:06 vm1 stonith-ng[14870]:   notice: stonith_topology_next: All fencing options to fence vm3 for crmd.14874@vm1.d5bd243d failed
Oct 15 15:18:06 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_fence reply from vm1: OK (0)
Oct 15 15:18:06 vm1 stonith-ng[14870]:    error: remote_op_done: Operation reboot of vm3 by vm1 for crmd.14874@vm1.d5bd243d: Generic Pacemaker error
Oct 15 15:18:06 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_notify reply from vm1: OK (0)
Oct 15 15:18:06 vm1 crmd[14874]:   notice: tengine_stonith_callback: Stonith operation 3/20:3:0:cffe5b98-3c92-4ed3-8992-426ef00df4ed: Generic Pacemaker error (-201)
Oct 15 15:18:06 vm1 crmd[14874]:   notice: tengine_stonith_callback: Stonith operation 3 for vm3 failed (Generic Pacemaker error): aborting transition.
Oct 15 15:18:06 vm1 crmd[14874]:     info: abort_transition_graph: tengine_stonith_callback:463 - Triggered transition abort (complete=0) : Stonith failed
Oct 15 15:18:06 vm1 crmd[14874]:   notice: tengine_stonith_notify: Peer vm3 was not terminated (reboot) by vm1 for vm1: Generic Pacemaker error (ref=d5bd243d-da15-4098-80ab-c9f1bce3827f) by client crmd.14874
Oct 15 15:18:06 vm1 crmd[14874]:   notice: run_graph: Transition 3 (Complete=1, Pending=0, Fired=0, Skipped=5, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-warn-0.bz2): Stopped
Oct 15 15:18:08 vm1 crmd[14874]:     info: crm_timer_popped: New Transition Timer (I_PE_CALC) just popped (2000ms)
Oct 15 15:18:08 vm1 crmd[14874]:     info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]
Oct 15 15:18:08 vm1 crmd[14874]:     info: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED
Oct 15 15:18:08 vm1 cib[14869]:     info: cib_process_request: Completed cib_query operation for section 'all': OK (rc=0, origin=local/crmd/70, version=0.8.25)
Oct 15 15:18:08 vm1 pengine[14873]:     info: process_pe_message: Input has not changed since last time, not saving to disk
Oct 15 15:18:08 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm2 is active
Oct 15 15:18:08 vm1 pengine[14873]:     info: determine_online_status: Node vm2 is online
Oct 15 15:18:08 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm3 is active
Oct 15 15:18:08 vm1 pengine[14873]:     info: determine_online_status: Node vm3 is online
Oct 15 15:18:08 vm1 pengine[14873]:     info: determine_online_status_fencing: Node vm1 is active
Oct 15 15:18:08 vm1 pengine[14873]:     info: determine_online_status: Node vm1 is online
Oct 15 15:18:08 vm1 pengine[14873]:  warning: unpack_rsc_op_failure: Processing failed op monitor for pDummy on vm3: not running (7)
Oct 15 15:18:08 vm1 pengine[14873]:  warning: pe_fence_node: Node vm3 will be fenced because of resource failure(s)
Oct 15 15:18:08 vm1 pengine[14873]:     info: native_print: pDummy	(ocf::pacemaker:Dummy):	FAILED vm3 
Oct 15 15:18:08 vm1 pengine[14873]:     info: group_print:  Resource Group: gStonith3
Oct 15 15:18:08 vm1 pengine[14873]:     info: native_print:      f1	(stonith:external/libvirt):	Started vm1 
Oct 15 15:18:08 vm1 pengine[14873]:     info: native_print:      f2	(stonith:external/ssh):	Started vm1 
Oct 15 15:18:08 vm1 pengine[14873]:     info: get_failcount_full: pDummy has failed 1 times on vm3
Oct 15 15:18:08 vm1 pengine[14873]:  warning: common_apply_stickiness: Forcing pDummy away from vm3 after 1 failures (max=1)
Oct 15 15:18:08 vm1 pengine[14873]:     info: RecurringOp:  Start recurring monitor (10s) for pDummy on vm1
Oct 15 15:18:08 vm1 pengine[14873]:  warning: stage6: Scheduling Node vm3 for STONITH
Oct 15 15:18:08 vm1 pengine[14873]:   notice: native_stop_constraints: Stop of failed resource pDummy is implicit after vm3 is fenced
Oct 15 15:18:08 vm1 pengine[14873]:   notice: LogActions: Recover pDummy	(Started vm3 -> vm1)
Oct 15 15:18:08 vm1 pengine[14873]:     info: LogActions: Leave   f1	(Started vm1)
Oct 15 15:18:08 vm1 pengine[14873]:     info: LogActions: Leave   f2	(Started vm1)
Oct 15 15:18:08 vm1 pengine[14873]:  warning: process_pe_message: Calculated Transition 4: /var/lib/pacemaker/pengine/pe-warn-0.bz2
Oct 15 15:18:08 vm1 crmd[14874]:     info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Oct 15 15:18:08 vm1 crmd[14874]:     info: do_te_invoke: Processing graph 4 (ref=pe_calc-dc-1381817888-54) derived from /var/lib/pacemaker/pengine/pe-warn-0.bz2
Oct 15 15:18:08 vm1 crmd[14874]:   notice: te_fence_node: Executing reboot fencing operation (20) on vm3 (timeout=60000)
Oct 15 15:18:08 vm1 stonith-ng[14870]:   notice: handle_request: Client crmd.14874.4eb0ff33 wants to fence (reboot) 'vm3' with device '(any)'
Oct 15 15:18:08 vm1 stonith-ng[14870]:   notice: initiate_remote_stonith_op: Initiating remote operation reboot for vm3: 61dd2280-87df-4984-9bb7-d573c83bd42f (0)
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_fence from crmd.14874: Operation now in progress (-115)
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query from vm1: OK (0)
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action list for agent fence_legacy (target=(null))
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action list for agent fence_legacy (target=(null))
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: process_remote_stonith_query: Ignoring reply from vm3, hosts are not permitted to commit suicide
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query reply from vm3: OK (0)
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: process_remote_stonith_query: Query result 2 of 3 from vm2 (2 devices)
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: call_remote_stonith: Total remote op timeout set to 120 for fencing of node vm3 for crmd.14874.61dd2280
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: call_remote_stonith: Requesting that vm2 perform op reboot vm3 with f1 for crmd.14874 (72s)
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query reply from vm2: OK (0)
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: dynamic_list_search_cb: Refreshing port list for f2
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: dynamic_list_search_cb: Refreshing port list for f1
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: process_remote_stonith_query: Query result 3 of 3 from vm1 (2 devices)
Oct 15 15:18:08 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_query reply from vm1: OK (0)
Oct 15 15:18:15 vm1 root: Mark:pcmk:1381817895
Oct 15 15:18:17 vm1 stonith-ng[14870]:   notice: process_remote_stonith_exec: Call to f1 for vm3 on behalf of crmd.14874@vm1: Generic Pacemaker error (-201)
Oct 15 15:18:17 vm1 stonith-ng[14870]:     info: call_remote_stonith: Requesting that vm1 perform op reboot vm3 with f2 for crmd.14874 (72s)
Oct 15 15:18:17 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_fence reply from vm2: OK (0)
Oct 15 15:18:17 vm1 stonith-ng[14870]:     info: stonith_command: Processed st_fence from vm1: Operation now in progress (-115)
Oct 15 15:18:17 vm1 stonith-ng[14870]:     info: stonith_action_create: Initiating action reboot for agent fence_legacy (target=vm3)
Oct 15 15:18:28 vm1 cib[14869]:     info: crm_client_new: Connecting 0x14d7ba0 for uid=0 gid=0 pid=17339 id=be5d31fc-c441-4d0e-819a-298e17eca0a7
